| Data and Documentation | Chronology and News | Program Status | Publications | Related Data and Resources | Contact Points |
ICOADS Web information page (Tuesday, 17-May-2011 22:02:20 UTC):
Release 2.5 1970-2007 Outstanding Data Questions
Note: This page represents work in progress. More detailed discussion will be added at a later date (tables defining deck and source ID are available here).
Figure 1a. Mislocated S Hemisphere duplicates discovered by Jack Woollen. For September 1983-January 1984 any report from deck 926 in the Southern Hemisphere containing an ID with first two characters "TF" was deleted.
Figure 1b. Pacific ship track moved 100 degrees east. For December 1983 any report from deck 732 containing an ID "TF" was deleted.
Table 1. Input numbers of platform type (PT) moored buoy (PT=6), drifting buoy (PT=7), and Coastal-Marine Automated Network (C-MAN; PT=13) reports for 1970-79, stratified by deck and source ID (DCK SID). One noteworthy data feature is grey shaded.
Table 2. As for Table 1, except for 1980-97.
Table 3a. As for Table 1, except for 1998-2004.
Table 3b. As for Table 1, except for 2005-2007.
Figure 2. Transient duplicate match information for 2000-04 (numbers of reports). Colors indicate match dup status (DUPS, ref. Fig. 3). On the abscissa, the lower numbers label the number of weather elements in common, and the upper numbers embedded within those categories label the number different. Noteworthy feature: the red bars e.g. indicate DUPS=6 (time/space only match). (Note: This is referred to as "transient" information because the final output DUPS can be changed through subsequent matches with all possible duplicates.)
Figure 3. Transient duplicate match information (ref. Fig. 2) for 2000-04 (percentage of reports) plotted for selected deck-to-deck matches. Noteworthly feature: the large percentage of DUPS=11 (time/space/ID only matches) for 792 vs. 874 (NCEP GTS ship vs. SEAS) matches (see also Example 6 below).
Example 1. Transient matches between decks 792 vs. 926 (NCEP FM 13 ship vs. IMMT). Noteworthy differences: ID in the first two cases; SST sign in the last two cases; cloud elements. (Note: these are random examples of deck-to-deck differences; whether any of them represent systematic patterns would require further detailed investigation. Field abbreviations, listed vertically, are defined here.)
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=3 DIFFER=0
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
200111 1 0 6080 3603NYT 340170******101522 43 56**** -20****7************************************00 ****106792103 5101*0 101************4
200111 1 0 6080 3601183FYT 34017098 22*101522 43 56**** -20****7**3**** 710*************************00NO**25106926112 5 11** 00400*0 20****14
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=7 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
200111 1 0 6530 730LFN 30 5797808210065**** 48**** 0 906634AA*******************************00 ****106792103 5101*0 004**** 1*****14
200111 1 0 6530 730121LF3N 30 5797808*100652 34 48**** 0 98663400*******************************00NO**25106926112 5 10** 00400*0 70****14
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=3 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
199811 1 0 7390 2250UCTZ 361200******102350 76************ 501**8*********************************32 **** 72792100 511000 101*0** 1*******
199811 1 0 7390 2250UCTZ 140 6098 22*10235**** 20******** 5099***********************************32RU**23 72926112 5 10** 00100** 70****1*
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=7 DIFFER=5
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2003 2 1 0 6020 35400UCUF 32015097 222100004 10 20******** 817745AA*******************************61 ****105792103 5111*0 101**** 1*****14
2003 2 1 0 6020 35400UCUF 30024097258*10023**** 20******** 88779500*******************************24RU**13105926112 5 11** 00100** 70****1*
MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=5 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2003 5 1 0 7480 35990DBLK 1013097****101412 6 -41**** -84 -9***5*********************************00 **** 69792103 5 11*0 101***********14
2003 5 1 0 7480 35990DBLK 1013097****101412 6 -41 55 -84 9***5*********************************00DE**14 69926112 5121** 0010030 70****44
MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=5 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2004 8 1 0 8260 35270DBLK 20011093**** 99903 1 -8**** -9 -18***1*********************************61 **** 33792103 5121*0 101***********14
2004 8 1 0 8260 35270DBLK 20011093**** 99903 1 -8 -8 -9 18***1*********************************61DE**14 33926112 5 11** 0010010 70****44
Example 2. Transient matches between decks 792 vs. 927 (NCEP FM 13 ship vs. US-keyed). Noteworthy differences: ID differences, cloud elements.
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=5 DIFFER=0
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2001 1 1 0 4400 18720DWCBPD 25011398**** 99467 23 70 40 0 6076457A** 5 525 6 6*******************63 ****160792103 5100*0 104**0* 1*****34
2001 1 1 0 4400 18720WCBP 25011398**** 99467 23 70 40 0 6076457A** 5 525 6 6**********AAAAA****63US****160927113 5 1000 104*00* 1*****34
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=5 DIFFER=0
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2003 1 1 0-1640 19710DELYJ9D 10 7798 211*****5 10 300 260 250****332420** 6 3 4 6 3*******************54 ****377792103 5101*0 104**0********14
2003 1 1 0-1640 19710ELYJ9 10 7798 211100905 10 300 260 250****332420** 6** 4 6 3**********AAAAA****54US****377927113 513100 104*00********14
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=2 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
200210 1 300 4440 27720VCBZ 190129*************** 240************4**9**** 3 2*************************83 ****169792103 5111*0 104************1
200210 1 300 4440 27720VCBZ 19012898 212********* 240************4*AAAA** 3 2****************AAAAA****83CA****169927113 5 1100 104*0*********14
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=7 DIFFER=6
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2001 3 1 300 4290 27760CG2960 210 2697 122100887 10 -2 -2 -2 10666500*******************************00 ****169792103 5 11*0 104**2* 0*****14
2001 3 1 300 4290 27760CG2960 280 7298 322101557 10 -36 -45 -7 0886400** 0 0****************AAAAA****00CA****169927113 511100 104*00* 0*****14
MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=5 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
1998 7 41200 1390 290308PAO 90 6298****101144 0 286******** 283522342** 7 3 8 7 411 4 2*************13 ****279792100 513000 104*0** 1*****24
1998 7 41200 1390 290308PAO 90 62981149101144 0 286 275 270 15541362** 9 2 1 9 2** 6 1****AAAAA****43CA****279927113 5 1100 104*30* 1*****24
MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=6 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
200412 1 0 1140 28440C6T2063 5012998 110100801 5 275 235 210****2*0909**** 6 5** 5*******************14 ****278792103 5 11*0 103**0********14
200412 1 0 1140 28440C6T2063 5012898 110100801 5 275 235 210****200909****** 5**************AAAAA****14US****278927113 512100 103*00********14
Example 3. Transient matches between decks 926 vs. 927 (IMMT vs. US-keyed). Noteworthy difference: ID.
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=4 DIFFER=0
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
200210 1 0 3710 29340WCZ553 80 9898****102214 0 225 175 140****44*6*0** 4 236 6 2*******************24US**42207926112 5 12** 0040200**0*****4
200210 1 0 3710 29340WCZ5535 80 9898****102214 0 225 175 140****44A6AA** 4**36 6 2**********AAAAA****24US****207927113 510200 104*00********34
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=7 DIFFER=0
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2003 5 1 0 4560 27480WZP816 40 4698505*10125**** 44******** 38******* 1 1*************************23US**46169926112 5 12** 00402** 10****14
2003 5 1 0 4560 27480WZP8164 40 4698505210125**** 44******** 38*AAAA** 1******************AAAAA****23US****169927113 510200 104*0** 1*****14
MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=5 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
1994 627 0 1750 11660ELLO 100 60** 21210105**** 295************6**9***********************************25 323261926 72 512100 101*0****0******
1994 627 0 1750 11660ELLO 100 6798 21210105**** 295************6*AAAA**********************AAAAA******CA****261927113 5 1100 104*0*********24
MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=7 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2002 8 1 0 4220 15590OWZU2 17012998 12*102101 10 180 160 150 1578085*0** 3 1 8 4 3*******************15US**41157926112 5122** 0040000 10****14
2002 8 1 0 4220 15590OWZU2 17012898 122102101 10 180 160 150 1578085AA** 3 1 8 4 3**********AAAAA****15US****157927113 5 1200 104*00* 1*****14
Example 4. Transient matches between decks 700 vs. 926 (VOSClim BUFR vs. IMMT). Noteworthy differences: Wind speed and visibility differences; omission of wave/swell elements from BUFR (or not decoded into IMMA).
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=6 DIFFER=2
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2003 424 0-2730 11360V2FM 140 4197152*10168**** 240 232 228****852230*******************************33 ****405700110 513100 103**0**********
2003 424 0-2730 11360V2FM 140 4096152*101683 6 240 232 228****852230** 5 122 9 4*******************33AU**21405926112 5111** 0000001 00****14
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=6 DIFFER=2
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2003 6 1 0 1590 12500MZHC8 21012998 12*10079**** 281 256 245****711406*******************************34 ****262700110 513100 103**0**********
2003 6 1 0 1590 12500MZHC8 21013097 12*10079**** 281 256 245****711406** 5 314 9 120 6 2*************34AU**21262926112 5111** 0000001 10****14
MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=7 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
200412 1 0 3200 18130MXBC6 18011098 22*10130**** 224 200 186 202630774*******************************65 ****196700110 512200 100**0**********
200412 1 0 3200 18130MXBC6 18010098 22*101308 20 224 200 186 202630774** 3 329 9 6*******************65GB**41196926112 5 12** 0000000 10****14
MATCH=12 CERTAIN: UNCERTAIN WEATHER ELEMENT WITH TIME/SPACE/ID COMMON=7 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
200412 1 0-2550 10540C6HS4 120 8297582*10170**** 182 172 165 220877220*******************************33 ****404700110 513000 103**0* 3*******
200412 1 0-2550 10540C6HS4 120 8296582*101703 10 182 172 165 220877220** 5 312 6 420 5 3*************33AU**21404926112 5120** 0030001 10****14
Example 5. Transient matches between decks 793 vs. 926 (NCEP FM 13 buoy vs. IMMT). Noteworthy differences: PT. Deck 793 PT=missing because these numbers are not legal 5-digit moored buoy numbers. Deck 926 was assigned PT=5 without regard to the IMMT Observation platform field (more work is needed in this area as IMMT can contain data from non-ship platform types).
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=6 DIFFER=0
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2000 8 1 600 5690 18062136 180 72961022101377 10 145 142 140************ 2 1*************************** ****142793103 6101*0 304**0********14
2000 8 1 600 5690 18061236 180 7296102*101377 10 145 142 140****7763**** 2 1*************************00GB**41142926112 5 11** 0040000**0****14
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=6 DIFFER=0
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2001 2 1 0-3840 101030102 290103955022100907 20 170 168 170******************************************* ****467793103**101*0 003**0********14
2001 2 1 0-3840 1010ELBX4 29010395502*100907 20 170**** 160****8874**** 5 4*************************23IL**13467926112 5 11** 00300*0 10****1*
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=6 DIFFER=0
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2002 3 1 600 1280 507003 110 4198 200101201 10 280 220 193************ 4 1*************************** ****255793103**101*0 004**0********14
2002 3 1 600 1280 5070DICB 110 4198 20*10120**** 280 220 194****422340** 4 1*************************64DE**11255926112 5 11** 0040001**0****14
MATCH=10 CERTAIN: CERTAIN WEATHER ELEMENT WITH TIME/SPACE COMMON=6 DIFFER=0
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2003 4 1 0 4590 274607620 110 2697 22*10148**** 17********************** 0*************************** ****169793103**101*0 004***********14
2003 4 1 0 4590 27460WCV762 110 2697 22*10148**** 17************8********* 0*************************00US**46169926112 5 11** 00402****0****14
Example 6. Transient matches between decks 792 vs. 874 (NCEP FM 13 ship vs. SEAS). Noteworthy differences: While these appear to be legitimate matches, many weather elements are different (e.g., large wind speed differences) or missing in SEAS. The SEAS data may need to be more carefully validated (see also Fig. 3).
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=1 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2002 9 11200 3850 14450OULL2 150 8295****101757 3 250******** 258***A*********************************55 ****192792103 5 11*0 104**** 1*****34
2002 9 11200 3850 14450OULL2 150285*********************** 175*****22A55**********************AAAAA****58 ****192874111 511100 304***********14
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=1 DIFFER=1
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
200210 8 600 6150 19120WBO3345 330 82**8172101292 19 25************8*AAAA** 3 435***********************82 **** 89792103 5 11*0 104***********14
200210 8 600 6150 19120WBO3345 330128***********8172******** 129 304822A82**********************AAAAA****** **** 89874111 511100 304**** 1*****14
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=7 DIFFER=6
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
2002 5 3 0-3550 17880P3RD8 25015496 111101350 5 180 150 130 195442500** 6 625 6 7*******************54 ****447792103 5 11*0 104**0* 1*****14
2002 5 3 0-3550 17880P3RD8 250925964200100050111 130**** 135 606422554****** 6**** 7*****150AAAAA****95 ****447874111 511100 304*3** 1*****14
MATCH=11 CERTAIN: TIME/SPACE/ID COMMON=7 DIFFER=6
DD WD
L LIIIIIIIII S P W D S S S S IIIII R B D S UU BP
Y M D H A ODDDDDDDDD V WWW L P A B P S NC CC W W W S S S D P HI ERCCCCC RTDV C COO 1 C I P PPTL IDWVITT SHTPWSII
R O Y R T N123456789 D W V W12 PA P T T T TNHLHMH D P H D P H 2 2 2S SS12345 RRSS 1 2SP 0 K D T SCII IIIITII IICBXXXR
200311 51800 4450 30390WCZ5528 320 6798 222103050 5 128 100 80 5066146A** 4 235 5 4*******************53 ****172792103 5 11*0 103**0* 1*****14
200311 51800 4450 30390WCZ5528 32065898616*100050222 75**** 305 402622453****** 5**** 4*****100AAAAA****50 ****172874111 511100 303*0** 1*****14
Figure 4. Deck 144 (TAO/TRITON and PIRATA Buoys) raw output as flagged by DUPS (numbers of reports). Noteworthy feature: Many data from the different SIDs appear to be unique (DUPS=0). SIDs 86 and 87 were deleted during final duplicate elimination preconditioning.
Figure 5. As for Figure 4, except for Deck 714 (Canadian Integrated Science Data Mgmt. Buoys). Noteworthy feature: SID 61 was often the best duplicate (DUPS=1) instead of SID 63. SIDs 61 and 62 were deleted during final duplicate elimination preconditioning.
Figure 6. As for Figure 4, except for Deck 780 (NODC/OCL World Ocean Database). Noteworthy feature: SIDs 88 and 89 were almost entirely worst duplicates although several DUPS=6 (time/space only matches). SIDs 88 and 89 were deleted during final duplicate elimination preconditioning.
Figure 7. As for Figure 4, except for Deck 883 (US National Data Buoy Center). Noteworthy feature: Excellent matching and selection as best duplicate (DUPS=1) of SID 122 vs. SIDs 50 and 121 except for missing SID 122 data in 1984, 1990-91. SID 122 was deleted during final duplicate elimination preconditioning.
Figure 8. As for Figure 4, except for Deck 667 (IATTC). Noteworthy feature: Although previously thought to be unique SID 71 sometimes duplicated itself. Any deck 667 report exactly identical to any other deck 667 report was deleted (see also Table 4).
Table 4. Deck 667 (IATTC). Noteworthy feature: 5,640,769 matches and only 28,442 duplicates. The uncommon disparity between the number of transient matches and total duplicates in 1980, 1984, and 1988 was indicative of multiple copies of single reports, while most of 1989 was duplicated (see also Fig. 8).
Figure 9. Deck 714 (ISDM drifters) showing reports with II=0 (ID present, but unknown type), PT=missing (see also Table 5).
Table 5. Deck 714 (ISDM drifters) listing: Left: the numbers with II=0 (see Fig. 9) or II=3 (WMO 5-digit buoy number) per year and SID; Right: illegal or unrecognizable buoy IDs associated with PT=missing (and II=0).
Figure 10. Deck 714 (ISDM drifters) comparing the QC flag settings (percentage of input reports) for position (1=good, 3=doubtful) between the earlier data versions (SIDs=61-62) and the latest data version (SID=63, used for Release 2.5). This plot is included for reference to show that an earlier QC problem was resolved (excessive flagging of position 1993-97, which lead to the flag data being "included," as opposed to "excluded" during the latest processing).
Table 6. Final Release 2.5 platform type (PT) versus sea surface temperature measurement method (SI) (blank for PT or SI signifies missing). A few suspicious, or noteworthy, patterns are highlighted. For the oceanographic platform types (PT= 10, 11, 12, 17, 18, 19, 20, 21), it should be noted that SI values such as bucket or intake not expected from the profile instrument types appear when the profile temperature value is unavailable or unusable, and a reference SST value is used instead.
Figure 11. Time evolution of the sea surface temperature measurement method indicator (SI) (see Table 8) per NCEP decks 792 (FM 13 ship), 793 (FM 13 buoy), 794 (FM 18 buoy), and 795 (C-MAN).
Table 7. NOAA/NCEP and Release 2.5 processing of the sea surface temperature measurement method indicator (SI) from original GTS messages (FM 13), into NCEP's BUFR format, and of FM 13 or NCEP BUFR data into IMMA format, for 1998-2007. Colon (":") signifies a translation between the different coding systems (ref. Table 8 below). Thus before 24 July 2002 NCEP incorrectly used the FM 13 SST indicator, instead of the BUFR equivalent. For ICOADS however, we were aware of the problem and corrected for it.
| SI | ||
|---|---|---|
| before 24 July 2002 | starting on 24 July 2002 | |
| BUFR | FM 13: FM 13 (incorrect) | FM 13: BUFR |
| IMMA | FM 13: IMMA | BUFR: IMMA |
Table 8. For reference, sea surface temperature measurement method indicator (SI) values currently available in four different observations formats (BUFR, IMMA0, IMMT-3, and FM 13), plus in the WMO Pub. 47 metadata format.
| BUFR | IMMA0 | IMMT-3 | FM 13 | WMO Pub. 47 |
|---|---|---|---|---|
| 0 Ship intake | 0 bucket | 0 Bucket thermometer | 0 Positive or 0 Intake | BTT Bait tanks thermometer |
| 1 Bucket | 1 condenser inlet (intake) | 1 Condenser inlet | 1 Negative Intake | BU Bucket thermometer |
| 2 Hull contact sensor | 2 trailing thermistor | 2 Trailing thermistor | 2 Positive or 0 Bucket | C Thermometer in condenser intake on steam ships, or inlet engine cooling system on motor ships |
| 3 Reversing Thermometer | 3 hull contact sensor | 3 Hull contact sensor | 3 Negative Bucket | HC Hull contact sensor |
| 4 STD/CTD sensor | 4 through hull sensor | 4 "Through hull" sensor | 4 Positive or 0 Hull contact sensor | HT "Through hull" sensor |
| 5 Mechanical BT | 5 radiation thermometer | 5 Radiation thermometer | 5 Negative Hull contact sensor | OT Other |
| 6 Expendable BT | 6 bait tanks thermometer | 6 Bait tanks thermometer | 6 Positive or 0 Other | RAD Radiation thermometer |
| 7 Digital BT | 7 others | 7 Others | 7 Negative Other | TT Trailing thermistor |
| 8 Thermistor chain | 8 unknown (applicable only to decks 705-707) | |||
| 9 Infra-red scanner | 9 unknown or non-bucket | |||
| 10 Micro-wave scanner | 10 implied bucket (an HSST SID or any match) | |||
| 11 Infrared radiometer | 11 reversing thermometer or mechanical sensor | |||
| 12 In line thermosalinograph | 12 electronic sensor | |||
| 13 Towed body | ||||
| 14 Other | ||||
| 15 Missing value |
Table 9. A small number (16) of Russian Marine Met. Data Set (MORMET; deck 732) examples were provided by Steve McIntyre (30 August 2010) from January 1973 near the equator in the west Pacific, where negative SST and AT are obviously unphysical, and where he concluded that the problems appeared likely to have arisen from mislocated reports (incorrect latitude/longitude). The data examined by Mr. McIntyre were from the previous Release 2.4. In this table instead we list the "Intermediate" output from the latest R2.5, i.e. containing flagged duplicates with a new data source included for R2.5: Russian Research Vessel (R/V) Digitization (deck 735). (Field abbreviations, listed vertically, are defined here.) Only four of the 16 MORMET reports survived duplicate elimination (see field DUPS, where e.g. 6 = worse duplicate: time/space match with ID mismatch) and based on comparison with the R/V data they do not appear to be spatially mislocated. Moreover the spurious SST and AT observations are all flagged by the trimming flags (SF and AF) as extreme (6 = less than 4.5 sigma lower limit). Additionally, wind speeds in MORMET also appear to be halved versus the R/V data (possible m/s to knots units translation problems), whereas locations and most other listed meteorological data elements (i.e. outside of temperatures and winds) between the two sources appear to match reasonably well. Figure 12 illustrates the locations of the MORMET (deck 732) data for January 1973, with colors indicating the SST trimming flag (SF) values. Figure 13 contrasts the data locations and SF values for all other decks. Figure 14 illustrates the MORMET locations, with colors indicating the wind trimming flag (UF and VF) values (note: the higher value of UF or VF is plotted, as that is how the data are trimmed). Figure 15 illustrates the R/V locations, with colors indicating the SST trimming flag (SF) values.
A W D DD
L L T N S P B WP D S D S UUSS AS
Y M D H A O ITTLDV I I I CD W V V WW L PI AT BT P S S NCH CC W W W S S S C I P PPQQSANN
R O Y R T N MCIISS D I D 1I DI WI V W1 PA PT TI TI T I TNHLIHMH D P H D P H K D T SCZAFFCC
1973 1 52100 -600 16500 0300 0220 6009715 10093 0 -21 35 -40 -197790500 6 5 411 8732 57 5 62156699
1973 1 52100 -600 16500 0200 10UMAY 0200111009715 10093 0 282 248 0 2857790500 6 5 411 8735 64 5 12 1111
1973 1 6 0 -660 16500 0200 10UMAY 02501 7009715 10094 0 272 244 0 2867690560 4 3 411 8735 64 5 12 1111
1973 1 6 0 -660 16500 0300 0250 4009715810094 0 -27 38 -42 -197590560 4 3 411 8732 57 5 62156699
1973 1 6 900 -810 16450 0300 0250 20098 2 10089 0 -24 41 -47 -166320501 1 9732 57 5 62156699
1973 1 6 900 -810 16450 0200 10UMAY 02501 30098 2 10089 0 276 236 0 2926320501 135 9735 64 5 12 1111
1973 1 61500 -950 16400 0300 0220 30098 2 10083 0 -20 48 -58 -155210601 2 8732 57 5 62146699
1973 1 61500 -950 16400 0200 10UMAY 02001 60098 2 10083 0 284 215 0 2935210601 235 8735 64 5 12 1111
1973 1131200 -730 15630 0300 0361 0098 2010110 0 -14 -1499 0 0 0732 57 5 0 146699
1973 1172100 -910 16500 0300 0360 10098 1 10105 0 -20 32 -36 -123320600 0 0 6 2732 57 5 62146699
1973 1172100 -910 16500 0200 10UMAY 03601 10098 1 10105 0 284 255 0 2983320600 0 013 6 2735 64 5 12 1111
1973 118 300 -800 16500 0300 0200 10099 3 10081 0 -9 29 -36 -86610600 0 9 4732 57 5 62156699
1973 118 300 -800 16500 0200 10UMAY 01801 10099 3 10081 0 304 255 0 3066610600 0 5 9 4735 64 5 12 1111
1973 118 900 -680 16500 0300 0130 1009715 10094 0 -21 31 -34 -76430501 0 4 4732 57 5 62156699
1973 118 900 -680 16500 0200 10UMAY 01201 2009715 10094 0 282 259 0 3076430501 0 4 4735 64 5 12 1111
1973 1181500 -560 16500 0300 0350 20098 2 10081 0 -19 33 -38 -105520500 1 4 4732 57 5 62156699
1973 1181500 -560 16500 0200 10UMAY 03501 40098 2 10081 0 286 252 0 3025520500 1 4 4735 64 5 12 1111
1973 125 0 -800 15580 0300 0 40 20097 2110070 0 -17 -67 03 9 3 1 0 0732 57 5 0 146699
1973 1251500 -590 16480 0300 0330 30098 1 10059 0 -21 36 -42 -132110640 2 3732 57 5 0 156699
1973 1252100 -700 16470 0300 0360 3009715 10066 0 -19 29 -33 -147630504 3 2 8 4732 57 5 62156699
1973 1252100 -700 16470 0200 10UMAY 03201 6009715 10066 0 286 261 0 2957630504 3 236 8 4735 64 5 12 1111
1973 126 300 -820 16430 0300 0330 30098 2 10045 0 -15 28 -32 -107420602 3 2 8 3732 57 5 62146699
1973 126 300 -820 16430 0200 10UMAY 03001 50098 2 10045 0 293 263 0 3027420602 3 236 8 3735 64 5 12 1111
1973 126 900 -940 16400 0300 0 60 2009725 10061 0 -23 31 -33 -117530504 1 3732 57 5 62146699
1973 126 900 -940 16400 0200 10UMAY 0 601 4009725 10067 0 278 260 0 3017530504 136 3735 64 5 12 1111
1973 1261500 -950 16400 0300 0 90 4009715 10041 0 -23 28 -29 -126530504 3 4732 57 5 62146699
1973 1261500 -950 16400 0200 10UMAY 0 901 8009715 10041 0 279 267 0 2996530504 336 4735 64 5 12 1111
1973 127 600 -900 15570 0300 0170 9009721610026 0 -21 -11887052 4 2 0 0732 57 5 0 146699
Figure 12. Locations of MORMET data for Jan. 1973, with colors indicating the SST trimming flag values: gray SF=1; green SF=2,3; blue SF=4,5; red SF=6,7; cyan SF=C. Following is an explanation of the trimming flags shown (see sec. 6 here for further information):
1 = within 2.8 sigma limits
2 = less than 2.8 sigma lower limit
3 = greater than 2.8 sigma upper limit
4 = less than 3.5 sigma lower limit
5 = greater than 3.5 sigma upper limit
6 = less than 4.5 sigma lower limit
7 = greater than 4.5 sigma upper limit
C = limits missing (ocean/coastal box)
A box (covering -9.5 to -5.6 latitude, and 155.7 to 165 longitude) has been inserted to mark the approximate region of the original 16 MORMET reports. Note suspicious data boxes, possibly indicating large blocks of mislocated data.
Figure 13. Locations of all non-MORMET decks for Jan. 1973, with colors indicating the SST trimming flag values: gray SF=1; green SF=2,3; blue SF=4,5; red SF=6,7; cyan SF=C (as for Fig. 12).
Figure 14. Locations of MORMET data for Jan. 1973, with colors indicating the wind trimming flag values (note: the higher value of UF or VF is plotted, as that is how the data are trimmed): gray UF/VF=1; green UF/VF=2,3; blue UF/VF=4,5; red UF/VF=6,7; cyan UF/VF=C (as for Fig. 12).
Figure 15. Locations of the R/V data for Jan. 1973, with colors indicating the SST trimming flag values: gray SF=1; green SF=2,3; blue SF=4,5; red SF=6,7; cyan SF=C (as for Fig. 12).
[Documentation and Software][Links to additional]
U.S. National Oceanic and Atmospheric Administration hosts the icoads website privacy disclaimer
Document maintained by icoads@noaa.gov
Updated: May 17, 2011 22:02:20 UTC
http://www.icoads.noaa.gov/pre2.5-2.html