top_n (and possibly other ranking functions) within group_by chains got really slow in the latest master branch version ( c7ca374 ).
Performance is fine when the number of groups is small, but it appears (in my light testing) that the slowdown is a (rapidly-growing) function of the number of groups.
Observe:
library(tidyverse); library(magrittr) n_groups <- 10e3 n_obs <- 100e3 x <- sample(n_groups, n_obs, TRUE) y <- runif(n_obs) df <- tibble(x, y) system.time(foo1 <- df %>% group_by(x) %>% top_n(1, y)) # user system elapsed # 4.868 8.328 13.195 system.time(foo2 <- df %>% group_by(x) %>% arrange(desc(y)) %>% mutate(ix = row_number()) %>% filter(ix == 1) %>% select(-ix)) # user system elapsed # 0.440 0.008 0.448 system.time(foo3 <- df %>% group_by(x) %>% arrange(desc(y)) %>% slice(1)) # user system elapsed # 0.112 0.000 0.113 identical(sort(foo1$y), sort(foo2$y)) ## TRUE identical(sort(foo2$y), sort(foo3$y)) ## TRUE
top_n performance previously appeared to be similar to the 'naive' foo2 and foo3 variants above.
Here's a performance analysis for n_groups varying between 1,000 and 10,000:
n_groups <- seq(1e3, 10e3, by = 1e3) n_obs <- 100e3 results <- lapply(n_groups, function(n) { print(n) df <- tibble(x = sample(n, n_obs, TRUE), y = runif(n_obs)) t1 <- system.time(df %>% group_by(x) %>% top_n(1, y)) t2 <- system.time(df %>% group_by(x) %>% arrange(desc(y)) %>% mutate(ix = row_number()) %>% filter(ix == 1) %>% select(-ix)) t3 <- system.time(df %>% group_by(x) %>% arrange(desc(y)) %>% slice(1)) lst(t1, t2, t3) }) %>% setNames(n_groups) map_df(results, function(x) map_df(x, "elapsed"), .id = "n_groups")
# A tibble: 10 x 4
n_groups t1 t2 t3
<chr> <dbl> <dbl> <dbl>
1 1000 1.663 0.201 0.122
2 2000 3.192 0.268 0.129
3 3000 4.038 0.226 0.099
4 4000 5.128 0.269 0.100
5 5000 6.328 0.297 0.104
6 6000 7.413 0.323 0.103
7 7000 8.737 0.358 0.107
8 8000 10.009 0.399 0.107
9 9000 11.118 0.418 0.111
10 10000 12.487 0.453 0.112