Common Locale Data Repository

Java locale data

• Java stores its locale data as Java classes in the package sun.text.resources
• Typical classes are
  • LocaleElements_en
  • BreakIteratorRules_th
  • DateFormatZoneData_en_IN
• These classes are private to Sun, and so can't be extended by others. If you have Java you can look at the source files
• Since they are in Java, they are not useful to other programming languages
• On the other hand, they can contain executable Java code to get exactly the right information

CLDR

• The CLDR is a collection of XML files containing l10n information
• The files are just labelled by their locale: en_AU.xml, fr_LU.xml, etc
• The DTD for this type is at http://www.unicode.org/cldr/dtd/1.2/ldml.dtd
• A sample document is fr_LU.xml

  
  <?xml version="1.0" encoding="UTF-8" ?>
  <!DOCTYPE ldml SYSTEM "http://www.unicode.org/cldr/dtd/1.2/ldml.dtd">
  <ldml >
  	<identity>
  		<version number="1.2"/>
  		<generation date="2004-08-27"/>
  		<language type="fr"/>
  		<territory type="LU"/>
  	</identity>
  	<numbers>
  		<symbols>
  			<group>.</group>
  		</symbols>
  		<currencies>
  			<currency type="FRF">
  				<symbol>FRF</symbol>
  			</currency>
  			<currency type="LUF">
  				<symbol>F</symbol>
  			</currency>
  		</currencies>
  	</numbers>
  </ldml>
        

• The home page is http://www.unicode.org/cldr/
• There doesn't seem to be any public Java API for handling these files - you can use the general Java XML API. But IBM's ICU4J uses CLDR to build its Java locale classes
• CLDR is also being used to generate POSIX l10n data files

Viewing CLDR file

The CLDR files are all encoded using UTF-8 (see a later lecture). A program to display these is

import javax.swing.*;
import java.awt.*;
import java.io.*;
import java.util.ResourceBundle;

public class UTFReader {

    public static ResourceBundle bundle;

    public static void main(String[] args) throws Exception {
	bundle = ResourceBundle.getBundle("UTFReader");

	if (args.length != 1) {
	    fatal("Usage");
	}
	
	new UTFReader(args);
    }
    
    public static void fatal(String errorType) {
	String errorMsg = bundle.getString(errorType);
	System.out.println("Fatal error: " + errorMsg);
	System.exit(1);
    }

    public UTFReader(String[] args) {
	FileInputStream fin = null;
	try {
	    fin = new FileInputStream(args[0]);
	} catch (FileNotFoundException e) {
	    fatal("FileNotFound");
	}
	
        BufferedReader in = null;
	try {
	     in = new BufferedReader(new InputStreamReader(fin, "UTF-8"));
	} catch (UnsupportedEncodingException e) {
	    fatal("NoEncoding");
	}


	JFrame frame = new JFrame();
	JTextArea text = new JTextArea(20, 100);
	JScrollPane pane = new JScrollPane(text);

	frame.setSize(600,600);
	frame.getContentPane().add(pane, BorderLayout.CENTER);
	frame.setVisible(true);

	String s = null;
	try {
	    while ((s = in.readLine()) != null) {
		text.append(s + "\n");
	    }
	} catch (IOException e) {
	    // do nothing	    
	}
    }
}

Usage=Usage: java UTFReader <filename>
FileNotFound: File not found
NoEncoding: Cannot decode file format

On my system, the following terminal will also allow UTF fonts to be viewed

LANG=en_GB.utf8 xterm -fn '-bitstream-bitstream cyberbit-medium-r-normal--0-0-0-0-c-0-iso10646-1'