ma_salmon maelle" />
+ - 0:00:00
Notes for current slide
Notes for next slide

Our package reviews in review!

Introducing & analyzing rOpenSci onboarding

Maëlle Salmon, editor at rOpensci onboarding
ma_salmon maelle

2018/03/17

1 / 79

Bonjour !

2 / 79

Bonjour !

library("magrittr")
# my chibi made by Oz Locke!
maelle <- magick::image_read("assets/happy_maelle.png") %>%
magick::image_resize("100x100")
magick::image_blank(500, 100, col = "salmon") %>%
magick::image_composite(maelle, offset = "+0+0") %>%
magick::image_annotate("Good morning! Goeie môre! Sanibonani! Molweni!",
boxcolor = "white",
location = "+100+25",
size = 15)

2 / 79

My place in the R community

row1 <- c(magick::image_read("assets/locke.png"),
magick::image_read("assets/rweekly.png")) %>%
magick::image_append()
row2 <- c(magick::image_read("assets/ropensci.png"),
magick::image_read("assets/rladies.png")) %>%
magick::image_append()
magick::image_blank(225, 200, col = "white") %>%
magick::image_composite(row1, offset = "+0+100") %>%
magick::image_composite(row2, offset = "+50+25")

3 / 79

Nancy, France

4 / 79

Nancy, France: La Place Stanislas

magick::image_read("assets/stan.JPG") %>%
magick::image_resize("400x400") %>%
magick::image_composite(maelle, offset = "+200+200")

5 / 79

Nancy, France: Le Parc de la Pépinière

magick::image_read("assets/pepiniere.JPG") %>%
magick::image_resize("400x400") %>%
magick::image_composite(maelle, offset = "+220+190")

6 / 79

Nancy, France: La Place Nelson Mandela

magick::image_read("assets/mandela.JPG") %>%
magick::image_resize("400x400")

7 / 79

What's rOpenSci?

8 / 79

What's rOpenSci?

  • Community of researchers and software developpers

  • R packages for open and reproducible science

  • Community and staff contributions

8 / 79

rOpenSci packages suite

https://ropensci.org/packages/

9 / 79

rOpenSci packages suite

https://ropensci.org/packages/

9 / 79

rOpenSci packages suite

https://ropensci.org/packages/

10 / 79

rOpenSci packages suite

https://ropensci.org/packages/

11 / 79

rOpenSci onboarding!

How to ensure quality in the whole suite?

12 / 79

rOpenSci onboarding!

How to ensure quality in the whole suite?

Open software reviews.

12 / 79

rOpenSci onboarding!

How to ensure quality in the whole suite?

Open software reviews.

  • drive adoption of best practices and standards
12 / 79

rOpenSci onboarding!

How to ensure quality in the whole suite?

Open software reviews.

  • drive adoption of best practices and standards

  • build a community of practice

12 / 79

rOpenSci onboarding!

How to ensure quality in the whole suite?

Open software reviews.

  • drive adoption of best practices and standards

  • build a community of practice

  • partnerships with the Journal of Open Source Software and Methods in Ecology and Evolution

12 / 79

What to review for?

13 / 79

What to review for?

magick::image_read("https://raw.github.com/jtleek/rpackages/master/documentation.png")

13 / 79

Review criteria

  • Open-source initiative (OSI) compatible license

  • Complete docs

  • High test coverage

  • Readable code

  • Usability

14 / 79

Review criteria

  • Open-source initiative (OSI) compatible license

  • Complete docs

  • High test coverage

  • Readable code

  • Usability

https://github.com/ropensci/onboarding/blob/master/packaging_guide.md

14 / 79

Review criteria

  • Open-source initiative (OSI) compatible license

  • Complete docs

  • High test coverage

  • Readable code

  • Usability

https://github.com/ropensci/onboarding/blob/master/packaging_guide.md

Soon as bookdown!

14 / 79

How to review?

  • Open & non-adversarial
15 / 79

How to review?

  • Open & non-adversarial

  • No rejections

15 / 79

How to review?

  • Open & non-adversarial

  • No rejections

  • Makes the process constructive for everyone involved

15 / 79

How to review?

  • Open & non-adversarial

  • No rejections

  • Makes the process constructive for everyone involved

  • Technically, using GitHub infrastructure

15 / 79

How to show stuff on GitHub?

16 / 79

How to show stuff on GitHub?

magick_webshot <- function(...){
webshot::webshot(...)
webshot <- magick::image_read("webshot.png")
file.remove("webshot.png")
webshot
}
16 / 79

rOpenSci onboarding

magick_webshot(url = "https://github.com/ropensci/onboarding",
selector = ".public", expand = c(92, 800, 500, 10))

17 / 79

Let's login chibimaelle

  • webshot docs: "If you are familiar with JavaScript"
18 / 79

Let's login chibimaelle

  • webshot docs: "If you are familiar with JavaScript"

  • Thanks Bea for coaching!

18 / 79

Let's login chibimaelle

  • webshot docs: "If you are familiar with JavaScript"

  • Thanks Bea for coaching!

  • Use the "return URL" instead of the URL

18 / 79

Let's login chibimaelle

magick_webshot(url = "https://github.com/login?return_to=%2Fropensci%2Fonboarding")

19 / 79

Let's login chibimaelle

magick_webshot_login <- function(url, ...){
return_url <- stringr::str_replace(url, "github.com/",
"github.com/login?return_to=%2F")
load("pwd.RData")
eval <- paste0("casper.then(function() {
this.sendKeys('#login_field', 'chibimaelle');
this.sendKeys('#password', '",pwd,"');
this.click('.btn-block');
this.wait(50);
});")
magick_webshot(return_url, eval = eval, ...)
}
20 / 79

Let's login chibimaelle

magick_webshot_login <- function(url, ...){
return_url <- stringr::str_replace(url, "github.com/",
"github.com/login?return_to=%2F")
load("pwd.RData")
eval <- paste0("casper.then(function() {
this.sendKeys('#login_field', 'chibimaelle');
this.sendKeys('#password', '",pwd,"');
this.click('.btn-block');
this.wait(50);
});")
magick_webshot(return_url, eval = eval, ...)
}
  • Only for my screenshots!

  • GitHub APIs (Application Programming Interfaces) to interact with GitHub and its data

20 / 79

rOpenSci onboarding, again!

magick_webshot_login("https://github.com/ropensci/onboarding",
selector = ".public", expand = c(0, 800, 500, 10))

21 / 79

The issues tracker

magick_webshot("https://github.com/ropensci/onboarding/issues",
selector = ".public", expand = c(0, 800, 500, 10))

22 / 79

Submitting a package

magick_webshot_login("https://github.com/AustralianAntarcticDivision/bowerbird",
selector = ".public", expand = c(0, 800, 500, 10))

23 / 79

Submitting a package

magick_webshot("https://github.com/ropensci/onboarding/issues",
selector = ".public", expand = c(0, 800, 500, 10))

24 / 79

Submitting a package

magick_webshot_login("https://github.com/ropensci/onboarding/issues/new",
selector = ".public", expand = c(0, 800, 500, 10))

25 / 79

Rejection

No rejections... but out-of-scope packages not onboarded.

https://github.com/ropensci/onboarding/blob/master/policies.md

26 / 79

Rejection

No rejections... but out-of-scope packages not onboarded.

https://github.com/ropensci/onboarding/blob/master/policies.md

  • fit in our categories: data retrieval, data extraction, database access, data munging, data deposition, reproducibility, geospatial data, text analysis.
26 / 79

Rejection

No rejections... but out-of-scope packages not onboarded.

https://github.com/ropensci/onboarding/blob/master/policies.md

  • fit in our categories: data retrieval, data extraction, database access, data munging, data deposition, reproducibility, geospatial data, text analysis.

  • application in science

26 / 79

Rejection

No rejections... but out-of-scope packages not onboarded.

https://github.com/ropensci/onboarding/blob/master/policies.md

  • fit in our categories: data retrieval, data extraction, database access, data munging, data deposition, reproducibility, geospatial data, text analysis.

  • application in science

  • better than similar packages

26 / 79

Rejection

No rejections... but out-of-scope packages not onboarded.

https://github.com/ropensci/onboarding/blob/master/policies.md

  • fit in our categories: data retrieval, data extraction, database access, data munging, data deposition, reproducibility, geospatial data, text analysis.

  • application in science

  • better than similar packages

When in doubt, pre-submission enquiry!

26 / 79

Pre-submission enquiry?

magick_webshot_login("https://github.com/ropensci/onboarding/issues/110",
selector = ".public", expand = c(0, 800, 500, 10))

27 / 79

Pre-submission enquiry?

magick_webshot_login("https://github.com/ropensci/onboarding/issues/129",
selector = ".public", expand = c(0, 800, 500, 10))

28 / 79

The review process

webshot::webshot("https://github.com/ropensci/onboarding/issues/139",
selector = "#show_issue")

29 / 79

The review process: editor checks

30 / 79

The review process: reviews

31 / 79

The review process: reviews

32 / 79

The review process

Ongoing discussion until acceptance and transfer

33 / 79

The review process

Ongoing discussion until acceptance and transfer

Often a blog post https://ropensci.org/tags/review/

33 / 79

A data-driven overview?

How to perform a data analysis of onboarding?

34 / 79

A data-driven overview?

How to perform a data analysis of onboarding?

Remember Jenny Bryan's talk last year?

34 / 79

A data-driven overview?

How to perform a data analysis of onboarding?

Remember Jenny Bryan's talk last year? Let's rectangle onboarding!

# Meme image by Allie Brosh https://en.wikipedia.org/wiki/Hyperbole_and_a_Half
magick::image_read("assets/rectangle.jpg") %>% magick::image_resize("300x300")

34 / 79

Onboarding data in the issue tracker

magick_webshot("https://github.com/ropensci/onboarding/issues",
selector = ".public", expand = c(0, 800, 500, 10))

35 / 79

Weaving GitHub issue threads

36 / 79

Weaving GitHub issue threads

GitHub GraphQL API v4. Better than v3? Get only the data you need.

36 / 79

Weaving GitHub issue threads

GitHub GraphQL API v4. Better than v3? Get only the data you need.

My experience

36 / 79

Weaving GitHub issue threads

GitHub GraphQL API v4. Better than v3? Get only the data you need.

My experience

36 / 79

GitHub API V4 explorer

37 / 79

ghql and what?

  • Use the query defined previously via ghql

  • Get raw JSON

  • writeClipboard

38 / 79

Learn how to transform JSON with jq PLAY!

39 / 79

A minimal V4 example

get_contents function of my ghrecipes package

40 / 79

A minimal V4 example: client creation

token <- Sys.getenv("GITHUB_GRAPHQL_TOKEN")
cli <- ghql::GraphqlClient$new(
url = "https://api.github.com/graphql",
headers = httr::add_headers(Authorization = paste0("Bearer ", token))
)
cli$load_schema()
41 / 79

A minimal V4 example: query writing

query <- paste0('query{
repository(owner: "', owner, '", name:"', repo,'"){
ref(qualifiedName: "master") {
target {
... on Commit {
id
history(first: 1) {
edges {
node {
tree{
entries {
name
}
}
}
}
}
}
}
}
}
}
')
42 / 79

A minimal V4 example: request\&wrangling

qry <- ghql::Query$new()
qry$query('foobar', query)
cli$exec(qry$queries$foobar) %>%
jqr::jq("..|.entries?|select(.!=null)|.[].name") %>%
as.character() %>%
stringr::str_replace_all('\\\"', '')
43 / 79

A minimal V4 example: in action

ghrecipes::get_contents(owner = "ropensci",
repo = "onboarding")
## [1] "CNAME" "README.Rmd"
## [3] "README.md" "editor_template.md"
## [5] "editors_guide.md" "icon_lettering_color.png"
## [7] "issue_template.md" "news_template.md"
## [9] "packaging.png" "packaging_guide.md"
## [11] "policies.md" "review_request_template.md"
## [13] "reviewer_template.md" "reviewing_guide.md"
44 / 79

Result: woven GitHub threads

GitHub V4 magic + some wrangling later...

45 / 79

Result: woven GitHub threads

GitHub V4 magic + some wrangling later...

threads <- readr::read_csv("data/clean_data.csv")
threads
## # A tibble: 2,521 x 10
## title created_at closed_at body user issue package
## <chr> <dttm> <dttm> <chr> <chr> <int> <chr>
## 1 rrli~ 2015-03-31 00:25:14 2015-04-13 23:26:38 "- 1~ rich~ 6 rrlite
## 2 rrli~ 2015-04-01 17:30:51 2015-04-13 23:26:38 "hey~ scko~ 6 rrlite
## 3 rrli~ 2015-04-01 17:36:03 2015-04-13 23:26:38 "@sc~ kart~ 6 rrlite
## 4 rrli~ 2015-04-02 03:36:09 2015-04-13 23:26:38 "Sur~ jero~ 6 rrlite
## 5 rrli~ 2015-04-02 03:50:43 2015-04-13 23:26:38 "IMO~ gabo~ 6 rrlite
## 6 rrli~ 2015-04-02 03:53:57 2015-04-13 23:26:38 "Ide~ rich~ 6 rrlite
## 7 rrli~ 2015-04-02 18:58:53 2015-04-13 23:26:38 "> H~ kart~ 6 rrlite
## 8 stpl~ 2015-04-08 23:56:17 2015-10-29 14:14:35 "- 1~ Robi~ 10 stplanr
## 9 rrli~ 2015-04-10 21:52:39 2015-04-13 23:26:38 "@ri~ stew~ 6 rrlite
## 10 rrli~ 2015-04-10 22:10:48 2015-04-13 23:26:38 "Tha~ rich~ 6 rrlite
## # ... with 2,511 more rows, and 3 more variables: is_review <lgl>,
## # commenter <chr>, role <chr>
45 / 79

Result: woven GitHub threads

length(unique(threads$package))
## [1] 70

46 / 79

Onboarding data in onboarded repos

47 / 79

Onboarding data in onboarded repos

magick_webshot_login("https://github.com/AustralianAntarcticDivision/bowerbird/commits/master",
selector = ".public", expand = c(0, 800, 500, 10))

47 / 79

Onboarding data in onboarded repos

48 / 79

Onboarding data in onboarded repos

48 / 79

Onboarding data in onboarded repos

48 / 79

Onboarding data in onboarded repos

48 / 79

Onboarding data in onboarded repos

48 / 79

Let the fun begin

  • Full code will be reported in blog posts.

  • Visualization packages: ggplot2, hrbrthemes, viridis.

49 / 79

How much work is onboarding?

50 / 79

Work done in repositories

51 / 79

Work done in repositories

52 / 79

Work done in repositories

53 / 79

Apparent age at submission

54 / 79

Apparent age at submission

54 / 79

How big are packages?

Work by authors

Work for reviewers

55 / 79

How big are packages? Lines of code

CRAN data via Bob Rudis.

56 / 79

How big are packages? Exports

CRAN data via Bob Rudis.

57 / 79

Reviewing time

58 / 79

Last notes on work quantification

59 / 79

Last notes on work quantification

  • Hard to define metrics
59 / 79

Last notes on work quantification

  • Hard to define metrics

  • Very hard working volunteers!

59 / 79

Last notes on work quantification

  • Hard to define metrics

  • Very hard working volunteers!

  • Decreasing time by automation

59 / 79

Last notes on work quantification

  • Hard to define metrics

  • Very hard working volunteers!

  • Decreasing time by automation

  • What about editors? In Tim Trice's words, "guiding angels from start to finish during the entire onboarding and review process".

59 / 79

A high-quality and... friendly process?

60 / 79

A high-quality and... friendly process?

rtweet::tweet_shot("801761258097803264") %>%
magick::image_crop("517x300+0+90")

60 / 79

A high-quality and... friendly process?

rtweet::tweet_shot("967074247301566465") %>%
magick::image_crop("517x300+0+90")

61 / 79

Social weather of onboarding

Ann Gentle's essay in http://open-advice.org/

Impressions, examples... more general approach for onboarding?

62 / 79

Social weather of onboarding

Ann Gentle's essay in http://open-advice.org/

Impressions, examples... more general approach for onboarding?

Tidy text analysis for the win! Reminder of Julia Silge's talk last year.

62 / 79

Social weather of onboarding

Ann Gentle's essay in http://open-advice.org/

Impressions, examples... more general approach for onboarding?

Tidy text analysis for the win! Reminder of Julia Silge's talk last year.

62 / 79

Most common words

63 / 79

Most common bigrams

64 / 79

Pairwise correlations

65 / 79

Sentiment

66 / 79

Sentiment

66 / 79

Words in negative lines

67 / 79

Negative sample

## # A tibble: 15 x 2
## line sentiment
## <chr> <dbl>
## 1 @ultinomics no more things, although do make sure to add mor~ -1.63
## 2 not sure what you mean, but i'll use different object names ~ -1.20
## 3 error in .local(.object, ...) : -1.00
## 4 error: -1.00
## 5 #### miscellaneous -1.00
## 6 error: command failed (1) -0.866
## 7 - get_plate_size_from_number_of_columns: maybe throwing an e~ -0.786
## 8 "this code returns an error, which is good, but it would be ~ -0.744
## 9 0 errors | 0 warnings | 0 notes -0.722
## 10 once i get to use this package more, i'm sure i'll have more~ -0.721
## 11 - i now realize i've pasted the spelling mistakes without th~ -0.707
## 12 minor issues: -0.707
## 13 ## minor issues -0.707
## 14 replicates issue -0.707
## 15 visualization issue -0.707
68 / 79

Most positive lines!

## # A tibble: 15 x 2
## line sentiment
## <chr> <dbl>
## 1 absolutely - it's really important to ensure it really has b~ 1.84
## 2 overall, really easy to use and really nicely done. 1.73
## 3 this package is a great and lightweight addition to working ~ 1.46
## 4 i am very grateful for your approval and i very much look fo~ 1.26
## 5 thank you very much for the constructive thoughts. 1.24
## 6 thanks for the approval, all in all a very helpful and educa~ 1.22
## 7 - really good use of helper functions 1.14
## 8 - i believe the utf note is handled correctly and this is ju~ 1.13
## 9 seem more unified and consistent. 1.13
## 10 very much appreciated! 1.13
## 11 - well organized, readable code 1.10
## 12 - wow very extensive testing! well done, very thorough 1.10
## 13 - i'm delighted that you find my work interesting and i'm ve~ 1.08
## 14 thank you very much for your thorough and thoughtful review,~ 1.08
## 15 great, thank you very much for accepting this package. i am ~ 1.07
69 / 79

Automate all the things

Let humans focus on what humans are best at!

70 / 79

Current automation

R CMD check/BiocCheck #repository standards
testthat::test_package() #functionality
covr::package_coverage() #testing completeness
devtools::spell_check() #documentation
lintr::lint_package() #code style
goodpractice::gp() #antipatterns/complexity

http://www.masalmon.eu/2017/06/17/automatictools/

71 / 79

More automation

72 / 79

More automation

  • Submission from R? Like devtools::release
72 / 79

More automation

72 / 79

More automation

72 / 79

More automation

  • Submission from R? Like devtools::release

  • Setting up the review project: https://github.com/ropenscilabs/pkgreviewr by Anna Krystalli

  • Matching reviewers and packages via better volunteering data

  • Running goodpractice::gp automatically and not locally

72 / 79

More automation

  • Submission from R? Like devtools::release

  • Setting up the review project: https://github.com/ropenscilabs/pkgreviewr by Anna Krystalli

  • Matching reviewers and packages via better volunteering data

  • Running goodpractice::gp automatically and not locally

And even more ideas!

72 / 79

Get involved!

rOpenSci

The official repo https://github.com/ropensci/onboarding

Read more https://ropensci.org/tags/review/

Volunteer to become a reviewer https://ropensci.org/onboarding/

73 / 79

Thank you!

Thank you and thanks to...

74 / 79

Thank you!

Thank you and thanks to...

images_annotate <- function(images, texts, ...){
purrr::map2(images, texts, magick::image_annotate, ...) %>%
magick::image_join()
}
set.seed(42)
thank <- function(names){
load("adjectives.RData")
compliments <- sample(adjectives, size = length(names))
glue::glue("https://github.com/{names}.png") %>%
magick::image_read() %>%
magick::image_resize("200x200") %>%
as.list() %>%
images_annotate(glue::glue("So {compliments}!"),
boxcolor = "white",
size = 15)%>%
magick::image_join() %>%
magick::image_append()
}
74 / 79

Thank you! satRday organizers

thank(c("DataWookie", "Axiematic", "jonmcalder", "theoniphotopoulou"))

thank(c("kakaalikins", "katrintirok", "kviljoen", "satrdays"))

75 / 79

Thank you! Feedback givers

thank(c("chucheria", "dacornu", "lauracion"))

76 / 79

Thank you! Onboarding editors

thank(c("karthik", "noamross", "sckott"))

77 / 79

Thank you! The faces of onboarding

Minus editors. Only threads of onboarded packages.

http://www.masalmon.eu/satrday_keynote/slides

78 / 79
79 / 79

Bonjour !

2 / 79
Paused

Help

Keyboard shortcuts

, , Pg Up, k Go to previous slide
, , Pg Dn, Space, j Go to next slide
Home Go to first slide
End Go to last slide
Number + Return Go to specific slide
b / m / f Toggle blackout / mirrored / fullscreen mode
c Clone slideshow
p Toggle presenter mode
t Restart the presentation timer
?, h Toggle this help
Esc Back to slideshow