It is the first meeting since the International Court of Justice awarded the Nigerian controlled Bakassi Peninsula to Cameroon last month.
The peninsula is estimated to hold about 10 billion barrels of oil. There have been repeated skirmishes over the years for the peninsula.
UN Secretary General Kofi Annan said he welcomed the renewed commitment by Nigerian president Olusegun Obasanjo and Cameroon's Paul Biya to 'renounce the use of force and pursue peaceful ways for the settlement of their boundary differences".
The mixed commission will consider ways of following up on the ICJ ruling. It will try to demarcate the land boundary, recommend additional confidence-building measures and oversee troop withdrawal.
The commission will hold its first meeting on December 1.