Enabling discoverability of results in R

1 So you have an awesome result …

1.1 That's great

No, really. Good results are hard to find.

2 Two questions

2.1 Is it reproducible?

2.2 How will someone find it?

1. Your collaborators now
2. You in a year

3 Let's talk Discoverability

3.1 Discoverability is

the ability to

• discover the existence of,
• locate, and
• retrieve in a useful form

Research, results, and computational artifacts

4 Super-flashy live demo time!

Note: Side-by-side R session and open RSS feed. plot a plot and see it show up 'magically' in the rss feed. Then search for a previous one and grab its code from the metadata stored in the db

5 How do we discover things now?

• Pubmed/Google Scholar/the library
• Figshare/Rpubs/Github + nbviewer
• Email authors/dig around on your hdd

6 It's all about the (meta)data

6.1 Figshare knows it

• Q: how discoverable is my research?
• A: It depends on what metadata you tag it with.

From Figshare FAQ (paraphrase)

6.2 Where does metadata come from?

6.3 That's tedious and you probably won't do it

I know

6.4 Sources of automatically inferable metadata

We can infer useful metadata about an object from the

1. computing environment
2. object
3. script
4. data being analyzed/transformed

6.5 The computing environment

• Who made the object
• When
• What packages were used/loaded

6.6 The object

R objects are necessarily self-describing

• names, dim, element classes on a data.frame
• levels on a factor
• Generally content of the object
  • think dput

6.7 The code

• Perfect low-level description of
  • how the object was made
  • what it is
• Often strong hints of
  • goal of analysis
  • thoughts/assumptions of analyst

6.8 The data

What

• data was analyzed?
• aspects of the dataset were used?
• transformations were applied

7 Plots

7.1 How we make plots

7.2 Plot design/choice tells us

• What analyst thought was important
• What relationships s/he was looking for

7.3 Plots

7.4 Plots

7.5 Plots

7.6 Flipping the script

7.7 Metadata about plots

• geom/plotting function used
• vars plotted
• conditioning/grouping vars
  • levels
• Titles, axis labels, legend, other text

8 Enough 'theory'! What does it do?

8.1 Store computational artifacts

In an

• Annotated
• Searchable
• Retrievable

Form

8.2 Automatically generate

Low-level semantic annotations

• Code
  • Tracked automatically
• Summaries of object

9 How do can you use it?

9.1 Mostly just go about your business

library(recordr)
##
## Your analysis code here
##
record(myplot)

9.2 Code tracking

• Code tracked automatically while pkg is loaded
  • All recorded objects have full code-provenance

9.3 Automatically recording objects

• autorecord function, records
  • all lattice/ggplot2 plots when drawn
  • any object passed to summary()

9.4 finding recorded objects

res = vtSearch("mtcars")

res2 = vtSearch("Awesome")

10 Sure, but why?

10.1 Collaboration/reporting

• RSS feed of plots/results in real time
• Searchable web frontend
• Distribution mechanism

10.2 Automatic annotation

• Plot.ly and figshare support tags
  • But don't generate them!

10.3 Organizational efficiency

• Has someone in my org already studied this data?
  • what did they find?
  • Maybe I should collab with them.
• And vice versa

10.4 Organizational safety

• I have a result and I'm about to make $100M bet on it
  • Are there 9 historical results that disagree?

10.5 Your own sanity

• Where the @$#% did I put that model/plot/etc I need for this paper?

11 History tracking

11.1 histry package

• Start tracking code you run when it is loaded
• R session or knitting documents

11.2 Demo

• History in a knitr document

12 Availability

12.1 Open source and on Github

• recordr: gmbecker/recordr
• histry: gmbecker/histry

13 Acknowledgements

• Sara Moore
• Michael Lawrence
• Biecek et al's archivist package