I’ve been getting a green screen playing back a few .mkv videos. The video stopped playing exactly after one minute and five seconds. I’ve reported this here, and I’ve tried a lot of silly things. The time I did this, I was using Nvidia 180.27 and the mplayer-vdpau-3402051.
Now, since the launchpad PPA repository for the nvidia-vdpau driver has been updated too, I’m on Nvidia driver version 180.29 and have also downloaded the new mplayer-vdpau-3482714.

The problem occurs again, only just a few secs earlier. But now I’m getting an error message:

Error 2 at libvo/vo_vdpau.c:1175

I’ve changes the value in libvo/vo_vdpau.c line 95 from 17 to 18, but it didn’t change anything.
But after I found the location again, where maximal reference frames are set (line 1075), I’ve set them onto the fix 16 again:

--- libvo/vo_vdpau.c.orig       2009-02-10 19:45:45.000000000 +0100
+++ libvo/vo_vdpau.c    2009-02-10 19:47:45.000000000 +0100
@@ -1073,7 +1073,7 @@
             break;
         case IMGFMT_VDPAU_H264_HIGH:
             vdp_decoder_profile = VDP_DECODER_PROFILE_H264_HIGH;
-            max_reference_frames = FFMIN(16, rndr->info.h264.num_ref_frames);
+            max_reference_frames = 16;
             break;
         case IMGFMT_VDPAU_VC1_SIMPLE:
             vdp_decoder_profile = VDP_DECODER_PROFILE_VC1_SIMPLE;

now it plays well, and I don’t have tearing either since I’m using the xorg.conf where composite is turned off ( nvidia-xconfig --no-composite )

If mplayer still keeps crashing with

Error 2 at libvo/vo_vdpau.c:1100

Try to increase increase line 96 in libvo/vo_vdpau.c step by step. In my case 19 works OK, even with KDE4.2 (composite and TwinView disabled):

#define NUM_VIDEO_SURFACES_H264 19 // (1 frame being decoded, up to 16 references)