Introduction to working with code lists

Jan van der Laan

The codelist package has an example code list and a data set that used codes from that code list. We will start by demonstrating how the package works using this example code list.

Let’s load the example code list:

> library(codelist)
> data(objectcodes)
> objectcodes
   code                  label parent locale missing
1     A                   Toys   <NA>     EN       0
2     B                  Tools   <NA>     EN       0
3   A01             Teddy Bear      A     EN       0
4   A02                Toy Car      A     EN       0
5   A03                Marbles      A     EN       0
6   B01                 Hammer      B     EN       0
7   B02         Electric Drill      B     EN       0
8     A              Speelgoed   <NA>     NL       0
9     B            Gereedschap   <NA>     NL       0
10  A01              Teddybeer      A     NL       0
11  A02          Speelgoedauto      A     NL       0
12  A03               Knikkers      A     NL       0
13  B01                  Hamer      B     NL       0
14  B02            Boormachine      B     NL       0
15    X         Unknown object   <NA>     EN       1
16    X Onbekend type voorwerp   <NA>     NL       1

We see that the code list contains codes for encoding various types of objects. A code list contains at the minimum a ‘code’ and ‘label’ column. The ‘code’ column can be any type; the ‘label’ column should be a character column. With the ‘parent’ column it is possible to define simple hierarchies. This columns should contain codes from the ‘code’ column. A missing value indicates a top-level code. With the ‘locale’ column it is possible to have different versions of the ‘label’ and ‘description’ (here missing) columns. It can be used for different translations as here, but could also be used for different versions of the labels and descriptions. The ‘missing’ column indicates whether or not the code should be treated as a missing value. This column should be interpretable as a logical column.

We will also load and example data set using the codes we loaded above:

> data(objectsales)
> objectsales |> head()
  product unitprice quantity totalprice
1     B01     70.65       67    4733.55
2     B01     76.93       76    5846.68
3     B01     43.49      100    4349.00
4     A03      3.08       26      80.08
5     A01     18.51       89    1647.39
6     A03      3.35       71     237.85

This is a data set containing the prices and sales of various products. The ‘product’ column uses codes from the objectcodes code list:

> objectsales$product |> head(10)
 [1] "B01" "B01" "B01" "A03" "A01" "A03" "A03" "B01" "A03" "A01"

One of the things we can do is convert the codes to their corresponding labels:

> to_labels(objectsales$product, objectcodes) |> head(10)
 [1] Hammer     Hammer     Hammer     Marbles    Teddy Bear Marbles   
 [7] Marbles    Hammer     Marbles    Teddy Bear
Levels: Toys Tools Teddy Bear Toy Car Marbles Hammer Electric Drill

The to_labels function accepts a vector with codes and a codelist for this vector. It can get a bit tiresome to keep having to pass in the codelist attribute. If it is missing, the looks for a ‘codelist’ attribute:

> attr(objectsales$product, "codelist") <- objectcodes
> to_labels(objectsales$product) |> head(10)
 [1] Hammer     Hammer     Hammer     Marbles    Teddy Bear Marbles   
 [7] Marbles    Hammer     Marbles    Teddy Bear
Levels: Toys Tools Teddy Bear Toy Car Marbles Hammer Electric Drill

The codelist package also has a code type. Converting to a code object adds the code class. This will result in some formatting and later on we will see that this also ensures that we cannot assign invalid codes to the vector:

> objectsales$product <- code(objectsales$product, objectcodes)
> objectsales$product |> head(10)
 [1] B01 B01 B01 A03 A01 A03 A03 B01 A03 A01
8 Codelist: A(=Toys) B(=Tools) A01(=Teddy Bear) A02(=Toy Car) ...X(=Unknown object)
> to_labels(objectsales$product) |> head(10)
 [1] Hammer     Hammer     Hammer     Marbles    Teddy Bear Marbles   
 [7] Marbles    Hammer     Marbles    Teddy Bear
Levels: Toys Tools Teddy Bear Toy Car Marbles Hammer Electric Drill

For code objects there is also the labels method:

labels(objectsales$product) |> head(10)

The labels method and the to_labels function can be used to get readable output from various R-functions:

> table(labels(objectsales$product), useNA = "ifany")

          Toys          Tools     Teddy Bear        Toy Car        Marbles 
             0              0             29             14             16 
        Hammer Electric Drill           <NA> 
            30              2              9 
> tapply(objectsales$unitprice, labels(objectsales$product), mean)
          Toys          Tools     Teddy Bear        Toy Car        Marbles 
            NA             NA      19.761034      12.432857       2.480625 
        Hammer Electric Drill 
     45.303000     205.350000 
> lm(unitprice ~ 0+labels(product), data = objectsales) 

Call:
lm(formula = unitprice ~ 0 + labels(product), data = objectsales)

Coefficients:
    labels(product)Teddy Bear         labels(product)Toy Car  
                       19.761                         12.433  
       labels(product)Marbles          labels(product)Hammer  
                        2.481                         45.303  
labels(product)Electric Drill  
                      205.350

By default codes that are considered missing are converted to NA when converting to labels. This can be prevented by setting the missing argument to FALSE:

> table(labels(objectsales$product, FALSE), useNA = "ifany")

          Toys          Tools     Teddy Bear        Toy Car        Marbles 
             0              0             29             14             16 
        Hammer Electric Drill Unknown object           <NA> 
            30              2              5              4

The droplevels removes unused codes from the levels of the generated factor vector:

> table(labels(objectsales$product, droplevels = TRUE), useNA = "ifany")

    Teddy Bear        Toy Car        Marbles         Hammer Electric Drill 
            29             14             16             30              2 
          <NA> 
             9

Locale

Using the ‘locale’ column of the code list it is possible to specify different versions of for the labels and descriptions. This can be used the specify different translations as in this example, but can also be used to specify different versions, for example, long and short labels. By default all methods will use the first locale in the code list as the defalult locale; the locale returned by the cl_locale function:

> cl_locale(objectcodes)
[1] "EN"

Most methods also have a locale argument with which it is possible to specify the preferred locale (the default is used when the preferred locale is not present). For example:

> labels(objectsales$product, locale = "NL") |> head()
[1] Hamer     Hamer     Hamer     Knikkers  Teddybeer Knikkers 
7 Levels: Speelgoed Gereedschap Teddybeer Speelgoedauto Knikkers ... Boormachine

It can become tedious having to specify the locale for each function call. The cl_locale will look at the CLLOCALE option, when present, to get the preferred locale. Therefore, to set a default preferred locale:

> op <- options(CLLOCALE = "NL")
> cl_locale(objectcodes)
[1] "NL"
> tapply(objectsales$unitprice, labels(objectsales$product), mean)
    Speelgoed   Gereedschap     Teddybeer Speelgoedauto      Knikkers 
           NA            NA     19.761034     12.432857      2.480625 
        Hamer   Boormachine 
    45.303000    205.350000 
> # Set the locale back to the original value (unset)
> options(op)

Looking up codes based on label

Using the codes function it is possible to look up the codes based on a set of labels. For example, below we look up the code for ‘Hammer’:

> codes("Hammer", objectcodes)
[1] "B01"

or getting the code list form the relevant variable itself using the cl method that returns the code list of the variable:

> codes("Hammer", cl(objectsales$product))
[1] "B01"

This could be used to make selections. For example, instead of

> subset(objectsales, product == "B02")
         product unitprice quantity totalprice
33 B02[Electri…]    284.85       52   14812.20
73 B02[Electri…]    125.85       73    9187.05

one can do

> subset(objectsales, product == codes("Electric Drill", cl(product)))
         product unitprice quantity totalprice
33 B02[Electri…]    284.85       52   14812.20
73 B02[Electri…]    125.85       73    9187.05

In general the latter is more readable and makes the intent of the code much more clear (unless one can assume that the people reading the code will now most of the product codes).

When comparing a code object to labels, it is also possible to use the as.label function. This will add the class “label” to the character vector. The comparison operator will then first call the codes function on the label:

> subset(objectsales, product == as.label("Electric Drill"))
         product unitprice quantity totalprice
33 B02[Electri…]    284.85       52   14812.20
73 B02[Electri…]    125.85       73    9187.05

This only works for the equal-to and not-equal-to operators.

Selecting this way has an advantage over selecting records based on character vectors or factor vectors. For example we could also have done the following:

> subset(objectsales, labels(product) == "Electric Drill")
         product unitprice quantity totalprice
33 B02[Electri…]    284.85       52   14812.20
73 B02[Electri…]    125.85       73    9187.05

However, a small, difficult to spot, spelling mistake would have resulted in:

> subset(objectsales, labels(product) == "Electric drll")
[1] product    unitprice  quantity   totalprice
<0 rows> (or 0-length row.names)

And we could have believed that no electric drills were sold. The codes function will also check if the provided labels are valid and if not will generate an error (the try is to make sure don’t actually throw an error).

> try({
+   subset(objectsales, product == codes("Electric drill", cl(product)))
+ })
Error in codes.default("Electric drill", cl(product)) : 
  Labels not present in codelist in current locale.

Since selecting on labels is a common operation, there is also the in_labels function that will return a logical vector indicating whether or not a code has a label in the given set:

> subset(objectsales, in_labels(product, "Electric Drill"))
         product unitprice quantity totalprice
33 B02[Electri…]    284.85       52   14812.20
73 B02[Electri…]    125.85       73    9187.05

This function will of course also generate an error in case of invalid codes.

> try({
+   subset(objectsales, in_labels(product, "Electric drill"))
+ })
Error in codes.default(labels, codelist) : 
  Labels not present in codelist in current locale.

In the examples above we used the base function subset, but this will of course also work within data.tables and the filter methods from dplyr.

Assignment of codes

When the vector with codes is transformed to a code object, it can of course also be assigned to:

> objectsales$product[10] <- "A01"
> objectsales$product[1:10] 
 [1] B01 B01 B01 A03 A01 A03 A03 B01 A03 A01
8 Codelist: A(=Toys) B(=Tools) A01(=Teddy Bear) A02(=Toy Car) ...X(=Unknown object)

Here the codes function can also be of use (again, an invalid label will result in an error so this is a safe operation):

> objectsales$product[10] <- codes("Teddy Bear", objectcodes)
> objectsales$product[1:10] 
 [1] B01 B01 B01 A03 A01 A03 A03 B01 A03 A01
8 Codelist: A(=Toys) B(=Tools) A01(=Teddy Bear) A02(=Toy Car) ...X(=Unknown object)

Another option is to use the as.label function which labels a character vector as a label:

> objectsales$product[10] <- as.label("Electric Drill")
> objectsales$product[1:10] 
 [1] B01 B01 B01 A03 A01 A03 A03 B01 A03 B02
8 Codelist: A(=Toys) B(=Tools) A01(=Teddy Bear) A02(=Toy Car) ...X(=Unknown object)

Hierarchies

Each code can have parent code. With this a simple hierarchy can be defined. At the top of the hierarchy are the codes without parent (NA). This is level 0. Codes with a parent in level 0 are in level 1 etc. Note that level 0 is a higher level than level 1. The example code list objectcodes has two levels:

> cl_nlevels(objectcodes)
[1] 2
> cl_levels(objectcodes)
 [1] 0 0 1 1 1 1 1 0 0 1 1 1 1 1 0 0

These levels can be used to ‘cast’ the codes to a higher level:

> objectsales$group <- levelcast(objectsales$product, 0)
> head(objectsales)
        product unitprice quantity totalprice    group
1 B01[Hammer]       70.65       67    4733.55 B[Tools]
2 B01[Hammer]       76.93       76    5846.68 B[Tools]
3 B01[Hammer]       43.49      100    4349.00 B[Tools]
4 A03[Marbles]       3.08       26      80.08 A[Toys] 
5 A01[Teddy B…]     18.51       89    1647.39 A[Toys] 
6 A03[Marbles]       3.35       71     237.85 A[Toys]

This is, for example, useful to create aggregates at higher levels. For example, we can calculate the total number of toys and tools sold:

> aggregate(objectsales[c("quantity", "totalprice")], 
+   objectsales[c("group")], sum)
        group quantity totalprice
1 A[Toys]         3274   43918.09
2 B[Tools]        1829  103011.65
3 X[Unknown…]      308   18184.42

Note that by default the code list of the vector returned by levelcast will be modified to only contain the codes in the higher hierarchy (this can be suppressed using the filter_codelist = FALSE argument):

> cl(objectsales$group)
   code                  label parent locale missing
1     A                   Toys   <NA>     EN   FALSE
2     B                  Tools   <NA>     EN   FALSE
8     A              Speelgoed   <NA>     NL   FALSE
9     B            Gereedschap   <NA>     NL   FALSE
15    X         Unknown object   <NA>     EN    TRUE
16    X Onbekend type voorwerp   <NA>     NL    TRUE

Also, when the data contains codes from different levels, trying to cast to a level lower than that some of the codes in the vector will result by default in an error. This can be controlled with the over_level argument.

Safety

Using a code vector also has the advantage that the codes assigned to will be validated against the code list, generating an error when one tries assign an invalid code:

> try({
+   objectsales$product[10] <- "Q"
+ })
Error in `[<-.code`(`*tmp*`, 10, value = "Q") : 
  Invalid codes used in value.

This makes a code object safer to work with than, for example, a character of numeric vector with codes (a factor vector will also generate a warning for invalid factor levels).

The codes function and the as.label function (which call the codes function) will also generate an error:

> try({
+   objectsales$product[10] <- as.label("Teddy bear")
+ })
Error in codes.default(value, codelist) : 
  Labels not present in codelist in current locale.

Assigning NA will of course still work:

> objectsales$product[10] <- NA

A code object is safer to work with than a factor vector. For example:

> x <- factor(letters[1:3])
> y <- code(1:3, data.frame(code = 1:3, label = letters[1:3]))

Comparing on invalid codes works with a factor while it will generate an error for code objects:

> try({ x == 4 })
[1] FALSE FALSE FALSE
> try({ y == 4 })
Error in Ops.code(y, 4) : Invalid codes used in RHS

The same holds when comparing on labels:

> try({ x == "foobar" })
[1] FALSE FALSE FALSE

A code cannot directly be compared on a label and will generate an error even when the label is valid:

> try({ y == "a" })
Error in Ops.code(y, "a") : 
  RHS not of the same class as the used codes of the LHS.

One should use either the codes or as.label function for that:

> try({ y == as.label("a") })
[1]  TRUE FALSE FALSE
> try({ y == as.label("foobar") })
Error in codes.default(e2, cl(e1)) : 
  Labels not present in codelist in current locale.