Peter McCawley is a Visiting Fellow in the Indonesia Project, Australian National University, and former Dean of the Asian Development Bank Institute, Tokyo.

As the events of the last few days have shown, it seems quite hard for Australian leaders to keep the bilateral relationship with Indonesia on a steady path.

Too often, our leaders wax enthusiastic about the relationship only to find themselves tongue tied a short time later when something goes wrong. And too often, Australia's relations with Indonesia swing between peaks and troughs.

There are various reasons for the ups and downs in the relationship but one main factor — glaringly evident during the past week — is the lack of understanding at the Australian end of the dynamics of political life in Jakarta. 

Just as controversial political issues swirl around and take on a life of their own in Canberra, so the same is true in Jakarta. And these days, political competition in Jakarta is intense.

Ever since the beginning of his second (and final) term in office in 2009, President Soesilo Bambang Yudhoyono's (SBY's) numerous political enemies have been exploring every possible angle they can think of to corner him — and hopefully even impeach him. Despite the fact that he has, in important ways, been a successful president, his critics have had great sport labelling him weak, hopeless, indecisive, and even cowardly.

It is within this context, and with an eye to an intense year of politics during 2014, that SBY simply cannot afford to look indecisive in view of the extensive publicity surrounding foreign intelligence operations in Indonesia.

Further, to exacerbate matters, a considerable amount of public comment at the Australian end has verged on being dismissive of Indonesian concerns. The line that, 'Shucks, they must have known,' has been very unhelpful. Add to this the Australian Government's insistence that there will be 'no comment' on intelligence matters and the Australian position appears quite uncompromising. 

President Obama handled his recent row with Chancellor Angela Merkel over a similar incident in a far more effective way: Obama moved quickly to distance himself from the activities of the US intelligence agencies. In addition, it was announced that several inquiries would be held. Obama's response helped provide Chancellor Merkel with the political space that she needed to contain the row within Germany. It would be helpful if Australian ministers could think of similar ways of helping President SBY contain the row in Jakarta.

So where do we go now, given that an awkward row has broken out between Indonesia and Australia? The outlook is not encouraging.

Looking ahead to 2014 and beyond, the chances that the bilateral relationship will become easier any time soon look dim.

For one thing, from the point of view of both Australia and Indonesia's ASEAN partners, SBY and his Vice President Boediono have provided excellent leadership in the region.It is most unfortunate that SBY and Boediono have found themselves in the midst of difficulties with Australia towards the end of their term in office. Both are careful and thoughtful leaders; both give high priority to tackling Indonesia's difficult internal problems; and have worked hard to strengthen regional cooperation across ASEAN. But their term in office ends next year. The outlook becomes much more uncertain beyond next October when there will be a new president in Indonesia.

For another thing, starting in early 2014 Indonesian political life will move into high gear.In the short-term — during most of 2014 — sharp political competition and turbulence seems certain to dominate the political scene in Indonesia.

A first round of noisy nation-wide elections will be held in April for over 20,000 seats in the national, provincial and district legislatures. Immediately after these parliamentary elections, national attention will focus on the first round of the presidential elections to be held in July. If no presidential candidate wins an absolute majority in the first round, then a second round will be held in September.

Even after the new president takes office in October, uncertainty will continue for months. New ministers and new parliamentarians will not settle into their jobs until well into 2015.

For most of this time the overwhelming focus of political life in Indonesia will be on domestic affairs. Leaders and the media will have scant time for anything other than the rapidly-changing political landscape within Indonesia. It is inevitable that most international issues — such as negotiations with Australia over asylum seekers — will rank low on Indonesia's list of priorities for most of 2014.

Further, looking beyond 2014 the challenges for Australia in working cooperatively with Indonesia are likely to multiply. Dennis Richardson, the former Secretary of the Department of Foreign Affairs and Trade and now Secretary of the Department of Defence, recently summarised the longer-term outlook that Australian faces in managing relations with Indonesia

[Another] big challenge is to continue to manage our near neighbourhood relations with deftness and sensitivity, consistent with our national interests. This is a challenge which confronts successive generations of Australian foreign policy advisers and decision makers. We have had mixed success. But the environment is changing. It is only a matter of time before we have a neighbour in Indonesia which has a bigger economy in nominal terms than our own. We are not used to that. As Indonesia grows wealthier and more confident it will become increasingly difficult for Australia to gain the attention of Indonesian decision makers to the extent that we think our interests might warrant. In other words, we may need to become more selective in what we push and what we ask for.

This, then, is the challenge that Australia faces in working cooperatively with Indonesia during the next few years and beyond — to find ways of setting clear priorities in the balance between what we push (such as our intelligence operations across the region) and what we ask for (such as cooperation in managing the flows of asylum seekers to Australia). As another former Secretary of the Department of Foreign Affairs and Trade, Richard Woolcott, noted earlier this year, Australia needs 'to make regular and improved consultations on a wide range of issues a policy habit, especially in advance of any major policy decision which we might take that could affect Indonesia.'

During the past few days, the job of working cooperatively with Indonesia has suddenly become a lot harder.