You want to start a research project – good! But you don't know where to start – no problem! Here is a simple workflow that will get you started in no time. It took me years of renaming and moving files, learning coding conventions, and plenty of trial and error (many errors!) to arrive at this simple but incredibly useful project setup. Make your life easier and start here. Harder things will come your way soon enough – I promise.
Start any R data analysis project with this standardized project structure. No need to wonder where data, files, and scripts should go every time you start a new project. There are surprisingly few differences across scientific disciplines for what a good project structure needs. In fact, this one structure is highly portable, intuitive and has proven super useful.
First, create your project folder manually and name it. Descriptive but short names are the most useful, e.g., "stress_and_brain_development". Next, create a new project in RStudio: File > New Project > Existing Directory and select the project folder.
Now run make_project.sh to give your project a solid structure to start from that looks like this:
project/
├── data/
├── documentation/
├── output/
│ ├── figures/
│ ├── tables/
│ └── results/
├── publications/
│ ├── manuscript/
│ ├── posters/
│ └── presentations/
├── R/
└── scripts/
Optionally, the script creates a .gitignore file configured to exclude sensitive data and system-specific files by default. This is to protect your workflow from unintentionally sharing sensitive materials with the world should you later decide to use Git for version control and host the project on GitHub. It's optional but I recommend creating it.
For details on the folder structure, see the explanation at the bottom of this page. None of this is set in stone. If you find your workflows require a different folder structure, you may adjust the script. What makes Bash scripts so useful is that they let you automate tasks that would otherwise be boring, time-consuming, and error-prone – and repeat them consistently whenever you need to.
A README is the first documentation file you should create and maintain from the very beginning. Its purpose is to tell other researchers working with your project (and your future self!) what the project is about, whom to contact with questions, and other high-level information they should know.
The script create_readme.sh creates a minimal README.md for a scientific R project. It interactively requests the project title and main contact information, then adds a few standardized sections on reproducibility and data availability. Existing README files are never overwritten without confirmation.
It can be a good idea to use a consistent and simple naming scheme across your entire workflow. I find it cleanest to use snake_case with lowercase letters whenever possible. This way, you never have to remember whether a particular word started with a capital letter or deal with inconsistent capitalization across files.
The script lowercase.sh takes care of the lowercase part for you. It recursively renames all files and folders within a directory to lowercase.
Before changing anything, the script performs a dry run showing you exactly what will be renamed and flagging name collisions. Common system metadata files such as .DS_Store and Thumbs.db are left alone.
It's a small thing, but consistent naming makes navigating files – and referring to them in code – a lot less annoying.
In disciplines that produce large data, like brain imaging, it's common for a pipeline to produce output for every patient (or every digging site, every chemical compound, ...) and you need to inspect the one file within each patient's folder that tells you how well the pipeline performed for example. Unfortunately, these files can be scattered across a deeply nested directory and clicking through each folder hierarchy would drive you (and anyone) insane. Why not pull out all the files matching a particular name or pattern and take a look at them? That's what collect.sh is for.
Tell the script where to look and what to look for, for example "*quality_report.pdf", and it collects the matching files from all subdirectories into a single flat folder on your Desktop. By default, all files are copied, so the originals stay exactly where they are.
The script automatically borrows enough information from the directory structure to make each filename unique, in case every one of your patients (digging sites, compounds, ...) has an output file that is identically named.
This script works particularly well with BIDS-style naming and directory structures. Identifiers such as sub-01, ses-02, modality, or acquisition labels aren't just useful for brain imaging datasets, they provide meaningful information whenever files need to travel outside their original directory. I therefore recommend adopting a BIDS-inspired naming convention for most of your files and workflows. To learn more about BIDS, go to: https://bids.neuroimaging.io/index.html (just don't go down the rabbit hole too hard, your project is waiting!)
A few short practical guides and notes that are aimed at making your life easier.
Contains useful templates.
If you would like to use the tools in this repo (you should, they make your researcher life easier!), follow the following steps:
- Clone this repository
- Make the desired script executable:
chmod +x create_project_structure.shThen run scripts like this:
./make_project.shThese scripts require Bash. If you're using macOS or Linux, you're ready to go. If you're on Windows, you'll need Git Bash.
- macOS: Use the built-in Terminal app.
- Linux: Use your regular terminal with Bash.
- Windows: Use Git Bash. It is included with Git for Windows. If you don't have it yet, install Git for Windows and make sure Git Bash is included during installation.
On Windows, project paths can be specified using Git Bash syntax, for example:
/c/Users/username/Documents/my_project
or:
C:/Users/username/Documents/my_project
The utilities do not initialize Git repositories or create RStudio projects. Create the project directory and RStudio project first, then run the desired utility.
If you've never used a Bash script before, don't worry—you don't need to learn Bash to use these utilities. Simply open the appropriate terminal, navigate to the directory containing the script you want to run, and follow the instructions above.
Always review generated files before initializing Git or pushing a project to a remote repository.
data/: Contains the data that is relevant to your analysis. If your analysis uses only part of a much larger dataset, I recommend keeping only the analysis-specific subset here. Adding timestamps to data filenames is additionally useful in case you regularly go back to the parent dataset to pull new observations.
documentation/: Contains any files relevant to understanding your project. This includes a codebook, variable keys, any supplementary materials like a protocol detailing how a measurement was made.
output/: Contains output generated by your analysis scripts. I divide this into figures/, tables/ for presentation-ready tables, and results/ for raw/analytical results.
publications/: This will contain material that summarizes your findings for sharing with other researchers. Usually, this folder stays empty for quite some time before you have something to put here (or forever haha). I found it useful to have a folder where I can put presentations of interim summaries or problems I need to discuss. posters come in once your project has matured and is ready for presentation at a conference. Finally, you may want to write up your project for publication in a peer-reviewed journal. I like to keep the manuscript together with the underlying data and scripts that generated its results.
R/: Contains custom R code your analysis scripts use. This can be complex functions, helper scripts, definitions of visual parameters like colour palettes, etc.
scripts/: Contains .R scripts that contain a full analysis or processing step. These scripts may source content from the R/ folder.