It appears that step names must be valid Python variable names. I tripped on this a few times while trying to give steps names with hyphens. It's not an onerous restriction, except the error message that returns is not very helpful (see below). Is it possible to validate step names before or during metaflow::run?
library(metaflow) #> Metaflow (R) 2.2.0 loaded #> Metaflow (Python) 2.2.3 loaded #> #> Attaching package: 'metaflow' #> The following object is masked from 'package:stats': #> #> step library(magrittr) metaflow("test_flow") %>% step(step = "start", next_step = "bad-step-name") %>% step(step = "bad-step-name", r_function = function(self) NULL, next_step = "end") %>% step(step = "end") %>% run() # Error in py_call_impl(callable, dots$args, dots$keywords) : # SyntaxError: invalid syntax (metaflowR.uppq6x7d, line 12) # # Detailed traceback: # File "/home/mdneuzerling/anaconda3/lib/python3.8/site-packages/metaflow/R.py", line 93, in run # module = imp.load_source('metaflowR', tmp.name) # File "/home/mdneuzerling/anaconda3/lib/python3.8/imp.py", line 171, in load_source # module = _load(spec) # File "<frozen importlib._bootstrap>", line 702, in _load # File "<frozen importlib._bootstrap>", line 671, in _load_unlocked # File "<frozen importlib._bootstrap_external>", line 779, in exec_module # File "<frozen importlib._bootstrap_external>", line 916, in get_code # File "<frozen importlib._bootstrap_external>", line 846, in source_to_code # File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed # Calls: <Anonymous> -> py_call_impl # Execution halted
Created on 2021-01-10 by the reprex package (v0.3.0)
Session info
devtools::session_info() #> ─ Session info ─────────────────────────────────────────────────────────────── #> setting value #> version R version 4.0.2 (2020-06-22) #> os Ubuntu 20.04.1 LTS #> system x86_64, linux-gnu #> ui X11 #> language en_AU:en #> collate en_AU.UTF-8 #> ctype en_AU.UTF-8 #> tz Australia/Melbourne #> date 2021-01-10 #> #> ─ Packages ─────────────────────────────────────────────────────────────────── #> ! package * version date lib source #> P assertthat 0.2.1 2019-03-21 [?] CRAN (R 4.0.0) #> callr 3.5.1 2020-10-13 [1] CRAN (R 4.0.2) #> cli 2.2.0 2020-11-20 [1] CRAN (R 4.0.2) #> P crayon 1.3.4 2017-09-16 [?] CRAN (R 4.0.0) #> P desc 1.2.0 2018-05-01 [?] CRAN (R 4.0.0) #> P devtools 2.3.2 2020-09-18 [?] CRAN (R 4.0.2) #> digest 0.6.27 2020-10-24 [1] CRAN (R 4.0.2) #> P ellipsis 0.3.1 2020-05-15 [?] CRAN (R 4.0.0) #> P evaluate 0.14 2019-05-28 [?] CRAN (R 4.0.0) #> P fansi 0.4.1 2020-01-08 [?] CRAN (R 4.0.0) #> P fs 1.5.0 2020-07-31 [?] CRAN (R 4.0.0) #> P glue 1.4.2 2020-08-27 [?] CRAN (R 4.0.0) #> P highr 0.8 2019-03-20 [?] CRAN (R 4.0.0) #> P htmltools 0.5.0 2020-06-16 [?] CRAN (R 4.0.0) #> jsonlite 1.7.2 2020-12-09 [1] CRAN (R 4.0.2) #> P knitr 1.30 2020-09-22 [?] CRAN (R 4.0.0) #> P lattice 0.20-41 2020-04-02 [?] CRAN (R 4.0.0) #> P lifecycle 0.2.0 2020-03-06 [?] CRAN (R 4.0.0) #> magrittr * 2.0.1 2020-11-17 [1] CRAN (R 4.0.2) #> P Matrix 1.2-18 2019-11-27 [?] CRAN (R 4.0.0) #> P memoise 1.1.0.9000 2020-05-09 [?] Github (hadley/memoise@4aefd9f) #> metaflow * 2.2.0 2020-12-19 [1] Github (netflix/metaflow@09ccb92) #> pkgbuild 1.2.0 2020-12-15 [1] CRAN (R 4.0.2) #> P pkgload 1.1.0 2020-05-29 [?] CRAN (R 4.0.0) #> P prettyunits 1.1.1 2020-01-24 [?] CRAN (R 4.0.0) #> processx 3.4.5 2020-11-30 [1] CRAN (R 4.0.2) #> ps 1.5.0 2020-12-05 [1] CRAN (R 4.0.2) #> P purrr 0.3.4 2020-04-17 [?] CRAN (R 4.0.0) #> R6 2.5.0 2020-10-28 [1] CRAN (R 4.0.2) #> P Rcpp 1.0.5 2020-07-06 [?] CRAN (R 4.0.0) #> P remotes 2.2.0 2020-07-21 [?] CRAN (R 4.0.0) #> P renv 0.12.3 2020-11-25 [?] CRAN (R 4.0.2) #> reticulate 1.18 2020-10-25 [1] CRAN (R 4.0.2) #> rlang 0.4.9 2020-11-26 [1] CRAN (R 4.0.2) #> P rmarkdown 2.6 2020-12-14 [?] CRAN (R 4.0.2) #> rprojroot 2.0.2 2020-11-15 [1] CRAN (R 4.0.2) #> P sessioninfo 1.1.1 2018-11-05 [?] CRAN (R 4.0.0) #> P stringi 1.5.3 2020-09-09 [?] CRAN (R 4.0.0) #> P stringr 1.4.0 2019-02-10 [?] CRAN (R 4.0.0) #> testthat 3.0.1 2020-12-17 [1] CRAN (R 4.0.2) #> usethis 2.0.0.9000 2020-12-20 [1] Github (r-lib/usethis@c1e8ed6) #> P withr 2.3.0 2020-09-22 [?] CRAN (R 4.0.0) #> P xfun 0.19 2020-10-30 [?] CRAN (R 4.0.2) #> P yaml 2.2.1 2020-02-01 [?] CRAN (R 4.0.0) #> #> [1] /[TRUNCATED]/renv/library/R-4.0/x86_64-pc-linux-gnu #> [2] /tmp/RtmpRhwoxA/renv-system-library #> [3] /tmp/Rtmpot7VVI/renv-system-library #> #> P ── Loaded and on-disk path mismatch.