huftis · GitHub

When the join column(s) of two data frames has the same name but different internal encodings, the dplyr join functions fail. Example:

library(dplyr)
load(file("http://huftis.org/nedlasting/r/dplyr-encoding-join-problem.Rdata"))

The two data frames can be joined by the ‘løpenummer’ column:

> str(d1)
Classes ‘tbl_df’ and 'data.frame':  6 obs. of  2 variables:
 $ løpenummer: int  1 2 3 4 5 6
 $ foo       : int  176 114 148 154 181 120
> str(d2)
Classes ‘tbl_df’ and 'data.frame':  6 obs. of  2 variables:
 $ løpenummer: int  4 5 6 7 8 9
 $ baz       : int  24 46 45 10 35 16

But actually trying to join them fails with an error message:

> d1 %>% left_join(d2, by="løpenummer")
Error: cannot join on columns 'løpenummer' x 'løpenummer': index out of bounds

The left_join() function does recognise that the data frames can be joined by ‘løpenummer’, but still fails:

> d1 %>% left_join(d2)
Joining by: "løpenummer"
Error: cannot join on columns 'l�penummer' x 'l�penummer': index out of bounds

Note that the ‘ø’ character is different in this error message. (On my system it’s shown as the ‘unknown character’ glyph.)

The two ‘løpenummer’ variables do have the exact same name, as confirmed by:

> names(d1)==names(d2)
[1]  TRUE FALSE

But the character encoding is different:

> Encoding(names(d1))
[1] "UTF-8"   "unknown"
> Encoding(names(d2))
[1] "latin1"  "unknown"

I observe this bug on R 3.2.2, with the latest Git version (2015-11-05) of dplyr, on a 64-bits Linux system (though the problem is also present in the released 0.4.3 version in R 3.2.1 on a Windows system).

Read the original on github.com ↗