Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions .rubocop_todo.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,6 @@
# Note that changes in the inspected code, or installation of new
# versions of RuboCop, may require this file to be generated again.

# Offense count: 1
# This cop supports safe autocorrection (--autocorrect).
Gemspec/RequireMFA:
Exclude:
- 'morph-cli.gemspec'

# Offense count: 8
# This cop supports safe autocorrection (--autocorrect).
# Configuration parameters: AllowMultipleStyles, EnforcedHashRocketStyle, EnforcedColonStyle, EnforcedLastArgumentHashStyle.
Expand Down
31 changes: 31 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).

## [Unreleased]

### Changed

- Require Ruby 3.3 or later (was Ruby 2.7)
- Replace unmaintained `rest-client` with `faraday` + `faraday-multipart`
- Replace unmaintained `archive-tar-minitar` with `minitar` 1.x
- Write the upload tarball to a temporary file instead of the shared `/tmp/out` path
- Load `~/.morph` config with safe YAML loading
- Fill in gem metadata: homepage, source code, bug tracker and changelog URIs; require
MFA for manual pushes to RubyGems

### Added

- SimpleCov coverage reporting and a much expanded test suite (CLI and HTTP behaviour
tested with WebMock, no network access in tests)

## [0.2.5] and earlier

No changelog was kept for releases up to and including 0.2.5. See the
[commit history](https://github.com/openaustralia/morph-cli/commits/main) for details.

[Unreleased]: https://github.com/openaustralia/morph-cli/compare/v0.2.5...HEAD
[0.2.5]: https://github.com/openaustralia/morph-cli/releases/tag/v0.2.5
60 changes: 43 additions & 17 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,41 +1,67 @@
[![Gem Version](https://badge.fury.io/rb/morph-cli.png)](http://badge.fury.io/rb/morph-cli)
[![Gem Version](https://img.shields.io/gem/v/morph-cli)](https://rubygems.org/gems/morph-cli)
[![CI](https://github.com/openaustralia/morph-cli/actions/workflows/ci.yml/badge.svg?branch=main)](https://github.com/openaustralia/morph-cli/actions/workflows/ci.yml)

# Morph Commandline
# Morph CLI

Runs Morph scrapers from the commandline.
Runs [morph.io](https://morph.io) scrapers from the command line.

Actually it will run them on the Morph server identically to the real thing. That means not installing a bucket load of libraries
and bits and bobs that are already installed with the Morph scraper environments.
Actually it will run them on the morph.io server identically to the real thing.
That means not installing a bucket load of libraries and bits and bobs that are
already installed with the morph.io scraper environments.

To run a scraper in your local directory
## Installation

You'll need Ruby 3.2 or later. Then

gem install morph-cli

## Usage

To run the scraper in your current directory

morph

Yup, that's it.

It runs the code that's there right now. It doesn't need to be checked into git or anything.
It runs the code that's there right now. It doesn't need to be checked into git
or anything.

The first time you run it, it will ask for your morph.io API key, which it
saves in `~/.morph`.

For help

morph help

## Installation
## Limitations

You'll need Ruby >= 1.9 and then
It uploads your code every time. So if it's big it might take a little while.
Scrapers are not usually so I'm hoping this won't really be an issue.

gem install morph-cli
It doesn't yet return you the resulting sqlite database (or use the one you
might have locally).

## Limitations
## Development

It uploads your code everytime. So if it's big it might take a little while. Scrapers are not usually so I'm hoping this won't really be an issue
After checking out the repo, run `bundle install` to install dependencies.
Then run the tests with

It doesn't yet return you the resulting sqlite database (or use the one you might have locally)
bundle exec rspec

## Contributing

Contributions are welcome! Please see the
[OpenAustralia Foundation contributing guide](https://github.com/openaustralia/.github/blob/main/.github/CONTRIBUTING.md)
for how we work: GitHub Flow, draft pull requests, signed-off commits (DCO)
and our contributor licence agreement.

1. Fork it
2. Create your feature branch (`git checkout -b my-new-feature`)
3. Commit your changes (`git commit -am 'Add some feature'`)
4. Push to the branch (`git push origin my-new-feature`)
5. Create new Pull Request
2. Create your feature branch (`git checkout -b feature/my-new-feature`)
3. Commit your changes (`git commit -s -am 'Add some feature'`)
4. Push to the branch (`git push origin feature/my-new-feature`)
5. Create a new pull request

## License

The gem is available as open source under the terms of the
[MIT License](LICENSE.txt).
16 changes: 13 additions & 3 deletions morph-cli.gemspec
Original file line number Diff line number Diff line change
Expand Up @@ -7,13 +7,23 @@ Gem::Specification.new do |spec|
spec.version = MorphCLI::VERSION
spec.authors = ["Matthew Landauer"]
spec.email = ["matthew@oaf.org.au"]
spec.description = "Command line interface for Morph"
spec.summary = "Command line interface for Morph"
spec.homepage = ""
spec.summary = "Run morph.io scrapers from the command line"
spec.description = "Command line interface for morph.io. Uploads the scraper in the " \
"current directory to a morph.io server, runs it there and streams " \
"the output back to your terminal."
spec.homepage = "https://github.com/openaustralia/morph-cli"
spec.license = "MIT"

spec.required_ruby_version = ">= 3.2"

spec.metadata = {
"homepage_uri" => "https://github.com/openaustralia/morph-cli",
"source_code_uri" => "https://github.com/openaustralia/morph-cli",
"bug_tracker_uri" => "https://github.com/openaustralia/morph-cli/issues",
"changelog_uri" => "https://github.com/openaustralia/morph-cli/blob/main/CHANGELOG.md",
"rubygems_mfa_required" => "true"
}

spec.files = `git ls-files`.split($INPUT_RECORD_SEPARATOR)
spec.executables = %w[morph]
spec.require_paths = ["lib"]
Expand Down
Loading