Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions Gemfile.lock
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,8 @@ GEM
docile (1.4.1)
json (2.6.3)
language_server-protocol (3.17.0.3)
mdtoc (0.3.1)
sorbet-runtime
method_source (1.0.0)
parallel (1.23.0)
parser (3.2.2.3)
Expand Down Expand Up @@ -69,6 +71,7 @@ GEM
simplecov-html (0.13.2)
simplecov-lcov (0.9.0)
simplecov_json_formatter (0.1.4)
sorbet-runtime (0.6.13169)
unicode-display_width (2.4.2)
yard (0.9.34)

Expand All @@ -77,6 +80,7 @@ PLATFORMS

DEPENDENCIES
bundler (~> 2.3)
mdtoc (~> 0.3.1)
pry (~> 0.14)
rake (~> 13.0)
record_loader!
Expand Down
126 changes: 83 additions & 43 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@

[![Ruby](https://img.shields.io/badge/ruby-3.0-e2242a?logo=ruby)](https://ruby-doc.org/3.0.6/)
[![Test Coverage](https://codecov.io/gh/sanger/record_loader/graph/badge.svg?token=AO7PUU5SB0)](https://codecov.io/gh/sanger/record_loader)
[![Current Version](https://img.shields.io/github/v/tag/sanger/record_loader?sort=semver&label=Release)](https://github.com/sanger/record_loader/tags)
[![Yard Docs](http://img.shields.io/badge/yard-docs-blue.svg)](https://rubydoc.info/github/sanger/record_loader)

RecordLoader provides a simple and standardized way of populating databases from information described in a series of
Expand All @@ -10,6 +11,27 @@ your production and development environments.

While written with ActiveRecord/Rails in mind, it is possible to use RecordLoader in different environments.

<!-- mdtoc -->
* [RecordLoader](README.md#recordloader)
* [Key features](README.md#key-features)
* [Installation](README.md#installation)
* [How to run](README.md#how-to-run)
* [How to generate new records loaders for your project (Rails)](README.md#how-to-generate-new-records-loaders-for-your-project-rails)
* [An example loader](README.md#an-example-loader)
* [Dev and Wip files](README.md#dev-and-wip-files)
* [RecordLoader Dependencies](README.md#recordloader-dependencies)
* [Triggering on deployment](README.md#triggering-on-deployment)
* [Within Sanger PSD](README.md#within-sanger-psd)
* [Non Rails Environments](README.md#non-rails-environments)
* [Development](README.md#development)
* [Setup](README.md#setup)
* [Testing](README.md#testing)
* [Releasing](README.md#releasing)
* [Contributing](README.md#contributing)
* [License](README.md#license)
* [Updating the table of contents](README.md#updating-the-table-of-contents)
<!-- mdtoc-end -->

## Key features

- Produce testable, reproducible data migrations across multiple environments
Expand Down Expand Up @@ -82,62 +104,62 @@ Suppose you want to create a loader to maintain a selection of product types. Yo

This will create several files:

#### `lib/tasks/record_loader.rake`
- `lib/tasks/record_loader.rake`

Adds the record_loader:all rake task which can be used to trigger all record loaders.
Adds the record_loader:all rake task which can be used to trigger all record loaders.

#### `lib/record_loader/application_record_loader.rb`
- `lib/record_loader/application_record_loader.rb`

Application specific base class for customization.
Application specific base class for customization.

#### `config/record_loader/product_types/default_records.yml`
- `config/record_loader/product_types/default_records.yml`

Example yaml file to begin populating with your record information. Record Loaders will load all yaml files from within
this directory, so it is possible to separate your records into multiple different files for better organization.
In addition yaml files ending in `.dev.yml` and `.wip.yml` exhibit special behaviour.
See [dev and wip files](#dev-and-wip).
Example yaml file to begin populating with your record information. Record Loaders will load all yaml files from within
this directory, so it is possible to separate your records into multiple different files for better organization.
In addition yaml files ending in `.dev.yml` and `.wip.yml` exhibit special behaviour.
See [dev and wip files](#dev-and-wip).

#### `lib/record_loader/product_type_loader.rb`
- `lib/record_loader/product_type_loader.rb`

The actual loader. It will look something like this:
The actual loader. It will look something like this:

```ruby
# frozen_string_literal: true
# This file was automatically generated via `rails g record_loader`

# RecordLoader handles automatic population and updating of database records
# across different environments
# @see https://rubydoc.info/github/sanger/record_loader/
module RecordLoader
# Creates the specified plate types if they are not present
class ProductTypeLoader < ApplicationRecordLoader
config_folder 'product_types'

def create_or_update!(name, options)
ProductType.create_with(options).find_or_create_by!(name: name)
```ruby
# frozen_string_literal: true
# This file was automatically generated via `rails g record_loader`

# RecordLoader handles automatic population and updating of database records
# across different environments
# @see https://rubydoc.info/github/sanger/record_loader/
module RecordLoader
# Creates the specified plate types if they are not present
class ProductTypeLoader < ApplicationRecordLoader
config_folder 'product_types'

def create_or_update!(name, options)
ProductType.create_with(options).find_or_create_by!(name: name)
end
end
end
end
```
```

The `config_folder` specifies which directory under `config/record_loader` will be used to source the yaml files.
The method `create_or_update!` will create the actual records, and should be idempotent (ie. calling it multiple times will
have the same effect as calling it once). `create_or_update!` will be called once for each entry in the yaml files,
with the first argument being the key, and the second argument being the value, usually a hash of options.
The `config_folder` specifies which directory under `config/record_loader` will be used to source the yaml files.
The method `create_or_update!` will create the actual records, and should be idempotent (ie. calling it multiple times will
have the same effect as calling it once). `create_or_update!` will be called once for each entry in the yaml files,
with the first argument being the key, and the second argument being the value, usually a hash of options.

#### `lib/record_loader/tasks/record_loader/product_type.rake`
- `lib/record_loader/tasks/record_loader/product_type.rake`

This contains the `record_loader:product_type` which will trigger the record loader, and also ensures that
`record_loader:product_type` will get invoked on calling `record_loader:all`.
This contains the `record_loader:product_type` which will trigger the record loader, and also ensures that
`record_loader:product_type` will get invoked on calling `record_loader:all`.

#### `spec/data/record_loader/product_types/product_types_basic.yml`
- `spec/data/record_loader/product_types/product_types_basic.yml`

A basic configuration for testing the loader. Tests use a separate directory to avoid coupling your specs to the data.
A basic configuration for testing the loader. Tests use a separate directory to avoid coupling your specs to the data.

#### `spec/lib/record_loader/product_type_loader_spec.rb`
- `spec/lib/record_loader/product_type_loader_spec.rb`

A basic rspec spec file for testing your loader. By default this just confirms that your loader creates the
expected number of records, and that it is idempotent.
A basic rspec spec file for testing your loader. By default this just confirms that your loader creates the
expected number of records, and that it is idempotent.

## Dev and Wip files

Expand Down Expand Up @@ -206,11 +228,20 @@ See {RecordLoader::Adapter} for information about custom adapters.

## Development

After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can
also run `bin/console` for an interactive prompt that will allow you to experiment.
### Setup

After checking out the repo, run `bin/setup` to install dependencies.

To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the
version number in `version.rb`, ensure the CHANGELOG.md is updated and that everything is committed.
### Testing

Then, run `bundle rspec` to run the unit tests.
You can also run `bin/console` for an interactive prompt that will allow you to experiment.
To install this gem onto your local machine, run `bundle exec rake install`.

### Releasing

To release a new version, update the version number in `version.rb`, ensure the CHANGELOG.md is updated
and that everything is committed.
Then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push
the `.gem` file to [rubygems.org](https://rubygems.org).

Expand All @@ -221,3 +252,12 @@ Bug reports and pull requests are welcome on GitHub at <https://github.com/sange
## License

The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

## Updating the table of contents

After updating this README, please update the table of contents.
To do this automatically with the [mdtoc](https://github.com/andornaut/mdtoc) gem, run:

```shell
bundle exec mdtoc -o README.md README.md
```
1 change: 1 addition & 0 deletions record_loader.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,7 @@ Gem::Specification.new do |spec|

# Development dependencies
spec.add_development_dependency 'bundler', '~> 2.3'
spec.add_development_dependency 'mdtoc', '~> 0.3.1'
spec.add_development_dependency 'pry', '~> 0.14'
spec.add_development_dependency 'rake', '~> 13.0'
spec.add_development_dependency 'rspec', '~> 3.12'
Expand Down
Loading