There is a growing concern about the extent to which science is replicable.  Psychologists have been unable to replicate some findings when they repeat past studies (Baker 2015).  In biomedical studies, the strain of organism or conditions in the lab can have much larger effects than were previously appreciated (e.g., Lithgow et al. 2018).  In ecology, large collaborative studies have demonstrated that what we often assume to be global patterns are surprisingly context dependent.  For example, plant ecologists often assume that there is a hump-shaped or concave-down relationship between productivity and diversity – there are more species at intermediate levels – but an assessment of 48 grassland sites around the world found this shape in only 1 site and 34 had non-significant patterns (Adler et al. 2011).  Similarly, nitrogen (N) is thought to be the resource that limits plant production in most terrestrial ecosystems.  However, when N, phosphorus (P), and potassium (K) were added experimentally to sites around the world, N alone had a significant effect at less than a quarter of the sites (10 of 42).  More sites (12) showed no response to fertilization with any nutrient, and even more sites (25) had a significant effect only if N and P were added together (Fay et al. 2015).

Several approaches have been developed to address replicability.  Psychologists have attempted to replicate published conclusions; the ‘Many Labs’ project replicated 10 of 13 well-known studies (Yong 2013) though a larger study could only reproduce 39 of 100 research findings (Baker 2015).  Groups have been formed, such as the Society for Open, Reliable, and Transparent Ecology and Evolutionary Biology (SORTEE).

Rather than collecting new data, another approach is to ensure that research is computationally reproducible (Alston & Rick 2021).  Fields such as engineering have established methods of independent validation and verification (IV&V), including the establishment of ‘shadow teams’ of scientists tasked with reproducing results (Raphael et al. 2020).  Other fields [economics?] require that a manuscript include several analyses of the same data as an indication of the robustness of the conclusions to the assumptions that underlie the analyses. 

Computational Reproducibility

An individual analysis requires hundreds of large and small decisions, and some of those decisions can dramatically affect the conclusions.  For example, Gould et al. (2025) invited teams of analysts to study the same dataset.  Nature published a summary of their paper which captures the conclusion succinctly: 246 biologists get different results from same data sets (Oza 2023).  Gould et al. compiled two datasets, one related to blue tits (Cyanistes caeruleus) and one to Eucalyptus.  Analyses were compared by expressing differences as standardized effect sizes, so that negative and positive values indicated negative and positive relationships, respectively, and statistical significance is determined by whether the confidence interval excludes zero.  The blue tit data were used to explore the question “To what extent is the growth of nestling blue tits influenced by competition with siblings?”.  Most analyses agreed that the effects were negative (mean effect size = -0.35), though effect sizes ranged from -1.55 to 0.38. Of the 131 individual analyses, 93 concluded that effects were significantly negative, 2 that effects were significantly positive, and 36 that there was no significant effect.  The Eucalyptus data were used to explore the question “How does grass cover influence Eucalyptus spp. seedling recruitment?”.  The conclusions of these analyses showed even more variability: the mean effect size was close to zero (-0.09) and effect sizes ranged from -4.47 to 0.39.  Of the 79 individual analyses, 15 concluded that effects were significantly negative, 11 that effects were positive, and 53 that there was no significant effect.  Even though teams analyzed the same data, they often came to contrasting conclusions based on decisions they made during analysis.

Computational reproducibility should be easier to demonstrate than reproducibility in lab environments.  Kitzes et al. (2018) state that “a research project is computationally reproducible if a second investigator (including you in the future) can recreate the final reported results of the project, including key quantitative findings, tables, and figures, given only a set of files and written instructions.”  Ideally, the script you produce for this course will fit this description!

The field of data science has much to teach us about computational reproducibility, including direct analyses of R code (McGowan et al. 2020).  Alston & Rick (2021) created a checklist of items that can be addressed to increase reproducibility.

Here, I focus on analysis organization, scripts, and workflow design.

Analysis Organization

Projects

RStudio uses projects to help organize analyses.  Specifically, a project is self-contained and provides access to data, scripts, etc.  We will follow this approach.  For additional details, see Robinson (2016) and Bryan (2017).

The main directory for the project is the working directory.  This is where R will automatically look for files, and where files that are produced will automatically be saved. This also provides portability, as the project file will be useable by a collaborator even if their directory structure differs from yours.

When you open a R project, the working directory is automatically set as the folder in which the project file is stored.  You can confirm the working directory using the getwd() function:
getwd()

Note: one advantage of using a R project is that the working directory is automatically set.  If needed, however, you can use the setwd() function to change the working directory. You can specify a particular folder by typing out the address, or you can use the choose.dir() function to open a window through which you can navigate to the desired folder.
setwd("C://Users/jbakker/Desktop/SEFS 502/")
setwd(choose.dir())

Note that the slashes that map out the folder hierarchy are the opposite of what is conventionally used in Windows.

Create a folder in which you will store all files associated with SEFS 502 class notes and examples.  Once you’ve created this folder, save a R project within it.  Use this project to open R and work through class examples.

Sub-folders

A key advantage of setting a working directory is that it allows a project to have a stable starting point yet to remain organized by having items in the appropriate sub-folders.  Commonly used sub-folders include:

• data
• graphics
• images
• library
• notes
• output
• scripts (note: most data scientists now would recommend that you store your scripts on GitHub. See ‘Workflow design’ below for more details)

From the working directory, you can navigate using the relative path of a file rather than its absolute path. Sub-folders are referenced by simply including them in the name of the file you are accessing. For example, if the file ‘data.csv’ was stored in the data sub-folder, we could open it like this:
dataa <- read.csv("data/data.csv", header = TRUE, row.names = 1)

Using relative paths keeps your code portable among machines and operating systems (Cooper & Hsing 2025).  However, note that you do need to have the same sub-folders in your project folder.

In the folder that contains your SEFS 502 class project, create the following sub-folders:

• data
• scripts
• functions
• graphics

You can also create additional sub-folders for class notes, readings, etc. if desired.

Project Options

RStudio allows you to save your history (record of code that was run previously) and the objects that were created earlier.  However, I advise against this because these shortcuts increase the chance of errors creeping in.  For example, objects created during earlier sessions are still available for manipulation even if the current version of the script does not create them.

To turn these options off in RStudio, go to the ‘Tools’ menu and select ‘Project Options’.  Under the ‘General’ settings, set these options to ‘No’:

• Restore .RData into workspace at startup
• Save workspace to .RData on exit
• Always save history (even if not saving .RData)

File Naming

Cooper & Hsing (2025) offer some principles for file naming (which I don’t follow consistently!):

• Avoid spaces and punctuation symbols, and use lower- and upper-cases consistently
• Use periods only to identify file types
• Use the underscore (‘_’) to distinguish elements within a string that will be separated later on.
• Use the dash (‘-‘) instead of a space
• Include informative descriptions of the file contents
• Use a word or phrase (a ‘slug’) to link outputs to the script that generated them.  For example, slug_data.csv, slug_analysis.R, and slug_results.csv could link files containing the data, analysis, and results of the same data.
• If including dates, use YYYY-MM-DD format to allow files to be easily sorted chronologically and to avoid confusion based on local dating conventions
• Start script names with a number to indicate their position within the analysis sequence
• If using numbers, pad single digit numbers with a zero so that they can be sorted properly.  In other words, use ’01’ rather than ‘1’.

Scripting

R Scripts

Scripting is one of the powerful aspects of R that distinguishes it from point-and-click statistical software.  When a script is clearly written, it is easy to re-run an analysis as desired – for example, after a data entry error is fixed or if you decide to focus on a particular subset of the data.  To capitalize on this ability, it is necessary to be able to manipulate your data in R.  New users often want to export their data to Excel, manipulate it there, and re-import it into R.  Resist this urge.  Working in R helps you avoid errors that can creep in in Excel – such as calculating an average over the wrong range of cells – and that are extremely difficult to detect (Broman & Woo 2018).

A script is simply a text file containing the code used (actions taken), along with comments explaining why those actions were taken.  I save these files with a ‘.R’ suffix to distinguish them from other text files.

Scripts can be created within RStudio’s editor pane (my preference) but could also be created in R’s Editor or even in a simple text editor like WordPad.  Commands have to be copied and pasted into R for execution.

A script can include many different elements, including:

• Data import
• Define functions that will be used elsewhere (later) in the script
• Data adjustments
• Analysis
• Graphing

In RStudio, you can organize your code into sections.  Sections can be inserted via the ‘Code -> Insert Section’ command, or you adding a comment that includes at least four trailing dashes (—-), equal signs (====), or pound signs (####).

You can use the ‘Jump to’ menu at the bottom of the script editor to navigate between sections.

Individual sections can be ‘folded’ (minimized) to make the overall structure more apparent. The code within that section is replaced with an icon consisting of a blue box with a bidirectional arrow in it.  This is also helpful because you can select and run the entire folded section simply by highlighting the symbol that icon.

If your script includes multiple sections:

• Alt-O minimizes all sections
• Alt-Shift-O maximizes all sections

Cooper & Hsing (2025) offer some guidelines for structuring scripts:

• Split long scripts so that each script does one thing.
• Include a standardized header at the top of the script that provides essential information about what it does, who created it, etc.
• Load libraries and data, and define global variables at the top of the script rather than throughout it.
• Use section headers to guide readers through the script.

The following figure provides an example script structure.

Interactive Notebooks (Jupyter, RMarkdown)

Interactive notebooks provide the capability to create a single document containing code, its output (statistical results, figures, tables, etc.), and narrative text.  They have been getting a lot of attention recently.  Key advantages are that the connections between data and output are explicit, and the output is automatically updated if the data change.

There are two broad types of interactive notebooks:

• Project Jupyter creates a file that contains the code, output, and narrative text.  This file is not of publication quality but is helpful for sharing an analysis with collaborators.  UW is testing opportunities to use JupyterHub for teaching; see the link in Canvas if you want to explore or use this.  Please note that this course’s JupyterHub workspace is only active for this quarter, so if you use it you will need to download any files that you wish to keep at the end of the quarter. 
• RMarkdown: A RMarkdown file consists of code and narrative text; Tierney (2020) provides a nice overview of RMarkdown and how to use it.  The code can include instructions to create figures or tables.  When this file is processed, the output specified by the code is produced and the resulting document is saved as a .pdf or .html file.  As such, this approach is closely aligned with publication – in fact, Borcard et al. (2018) state that their entire book was written using RMarkdown in RStudio!  (I considered creating these notes in RMarkdown but apparently it doesn’t integrate with the PressBooks publishing platform).

Workflow Design

Best Practices

Ecologists (myself included) are rarely trained as programmers, but knowing some of the ‘best practices’ that programmers use can improve the quality of our scripts.  Examples of these practices include:

• Modularizing code (i.e., writing custom functions) rather than copying and pasting it to apply it to different objects.  This prevents errors where you update the code in one location in your script but forget to update it elsewhere.  Modularizing code is also helpful for maintaining consistency.  For example, imagine that you want to create multiple figures with a consistent format.  If the script is written as a function, it can be called multiple times with different data.  Formatting details would remain the same from figure to figure.  And, if you decided to adjust the formatting you would only have to do so in one place!
• Verifying that code is working correctly via assertions and unit tests (automated testing of units of code)
• Version control to keep track of changes.  Many (most?) data scientists use Git for this purpose.  Closely associated with this is GitHub, which enables code sharing.  The data files for this book are available in a GitHub repository.

Cooper & Hsing (2025) is an excellent resource in this regard.  There is also a growing body of open-source resources (free on-line lessons; in-person workshops at various locations) available through Software Carpentry.  And, the ‘Happy Git with R’ website appears to be quite readable and useful.

Workflows can be described verbally or graphically, as in this example:

McCune & Grace (2002; ch. 8) provide a flowchart and tabular techniques that can assist in designing a workflow.  See this Appendix for a summary.

Formatting Code

Reproducibility requires that your code also be accessible to others.  For example, an object name might make sense to you but be meaningless to someone who isn’t familiar with your study system.

Some best practices and guidelines for formatting code are provided in Table 1 below.

Conclusions

The scientific process is built on the idea of reproducibility, both in the field and at the computer.  The ability to script an analysis is essential to ensuring that analyses can be shared and repeated.

References

Adler, P.B., E.W. Seabloom, E.T. Borer, H. Hillebrand, Y. Hautier, A. Hector, W.S. Harpole, L.R. O’Halloran, J.B. Grace, T.M. Anderson, et al. 2011. Productivity is a poor predictor of plant species richness. Science 333:1750-1753.

Alston, J.M., and J.A. Rick. 2021. A beginner’s guide to conducting reproducible research. Bulletin of the Ecological Society of America 102(2):e01801.

Baker, M. 2015. First results from psychology’s largest reproducibility test. Nature.

Borcard, D., F. Gillet, and P. Legendre. 2018. Numerical ecology with R. 2nd edition. Springer, New York, NY.

Broman, K.W., and K.H. Woo. 2018. Data organization in spreadsheets. The American Statistician 72(1):2-10.

Bryan, J. 2017. Project-Oriented Workflow.

Cooper, N., and P-Y. Hsing (eds.). 2025. Reproducible Code. British Ecological Society, London, UK.

Fay, P.A., S.M. Prober, W.S. Harpole, J.M.H. Knops, J.D. Bakker, E.T. Borer, E.M. Lind, A.S. MacDougall, E.W. Seabloom, P.D. Wragg, et al. 2015. Grassland productivity limited by multiple nutrients. Nature Plants 1:15080.

Filazzola, A., and C.J. Lortie. 2022. A call for clean code to effectively communicate science. Methods in Ecology and Evolution 13:2119-2128.

Gould, E., H.S. Fraser, T.H. Parker, S. Nakagawa, S.C. Griffith, P.A. Vesk, F. Fidler, D.G. Hamilton, R.N. Abbey-Lee, J.K. Abbott, et al. 2025. Same data, different analysts: variation in effect sizes due to analytical decisions in ecology and evolutionary biology. BMC Biology 23:35.

Kitzes, J., D. Turek, and F. Deniz (eds.). 2018. The Practice of Reproducible Research: Case Studies and Lessons from the Data-Intensive Sciences. University of California, Oakland, CA.

Konstantinoudis, G., V. Gómez-Rubio, M. Cameletti, M. Pirani, G. Baio, and M. Blangiardo. 2023. A workflow for estimating and visualising excess mortality during the COVID-19 pandemic. The R Journal 15(2):89-104.

Lithgow, G.J., M. Driscoll, and P. Phillips. 2018. A long journey to reproducible results. Nature 548:387-388.

McCune, B., and J.B. Grace. 2002. Analysis of ecological communities. MjM Software Design, Gleneden Beach, OR.

McGowan, L.D., S. Kross, and J. Leek. 2020. Tools for analyzing R code the tidy way. The R Journal 12:226-242.

Oza, A. 2023. Reproducibility trial: 246 biologists get different results from same data sets. Nature 622:677-678.

Raphael, M.P., P.E. Sheehan, and G.J. Vora. 2020. A controlled trial for reproducibility. Nature 579:190-192.

Robinson, A. 2016. icebreakeR.

Tierney, N. 2020. RMarkdown for Scientists.

Yong, E. 2013. Psychologists strike a blow for reproducibility. Nature.