Research renovate (dep management)

[bebraw]

renovate is a tool like Greenkeeper that might help with dependency management.

To quote @rarkins from email:

I think there's still a gap between what we everyone does today and any "perfect" solution for complicated dependencies like Webpack that have a lot of related dependencies, peer dependencies, etc. For example I've implemented a feature for "lazy" grouping where you can define a group pattern or list of dependency names that should be combined into one branch when updating, but if only one of them has an update currently then it can be tested and merged on its own if tests pass. Personally I use such groups for eslint, angular and jest but such groupings might be useful for Webpack consumers too.

BTW another nice feature of renovate w.r.t. Webpack is that by default it will maintain a separate branch/PR per major release, e.g. someone on 2.x today will not only have received a PR for Webpack 3.0 in a renovate/webpack-3.x branch but will also continue to receive updates in a renovate/webpack-2.x branch in the meantime. And if they're on 1.x .. you could get all three branches. If the user decides "I'm not upgrading from 1.x for a long time" then they can simply close/delete the unwanted PRs and renovate will "learn" and suppress all updates until the user reopens the PR or manually patches to 2.x or 3.x in future (the logic behind this is surprisingly simple).

I opened this issue so we can keep related discussion here in public.

[rarkins]

Thank you @bebraw for suggesting this. I've waited a little while before following this up as I wanted to enhance a few things and battle test Renovate first before being able to say sincerely that it's ready for a critical community project like Webpack.

About Renovate

Renovate is an open source dependency updater that is available either as a CLI tool to host yourself (supporting GitHub, GitLab, Microsoft VSTS, and soon BitBucket) and also run by me as a free GitHub App. Both use the same code and configuration so switching at any time is possible. Renovate currently supports maintaining npm/yarn package.json files, meteor package.js files, Dockerfile images, .travis.yml node versions, and bazel WORKSPACE files.

Currently, the GitHub App supports 1000+ projects and creates hundreds of PRs daily. Installed organizations include GoogleChromeLabs, mozilla, webcomponents, algolia and apollographql.

Renovate Philosophy

Everything is configurable

Dependency updating is not a place to be stubbornly opinionated. Instead, Renovate uses a cascading JSON config that allows you to configure behaviour down to an almost silly level. e.g. users can configure behaviour per-package file, per dependency-type (e.g. devDependencies, peerDependencies, etc). per-package (including regex patterns such as /^webpack*/), and per-upgrade type (e.g. major/minor/patch).

Hide complexity when possible

Renovate supports "preset" configs similar to eslint shared configs (using npm namespaces, etc). As the tool author, this allows me to define very flexible configuration language options inside the tool but then wrap it up in presets that are easier for user consumption. i.e. users get the benefits of complex flexibility while not imposing it unless they need it.

To date, most presets in use have been published by me, but the tool allows for "democratised" distribution via npm using the renovate-config-* namespace. Some users with a large number of repositories have already set up their own custom renovate-config-<name> packages that they use to avoid config duplication across projects.

Don't ask users to configure settings which can be deduced

Where possible, Renovate's default behaviour is to detect appropriate settings at runtime, while then allowing users to manually override whenever desired. The goal is minimising configuration while still providing flexibility.

Cut the noise

The benefits of automated dependency updates are obvious. But the increased noise can also be annoying. Renovate's goal is not to be in your face with a firehose of PRs, although this is essentially the default configuration (you will receive a separate PR for each dependency update usually within 1 hour of it being released to npmjs).

Renovate supports multiple ways to reduce the "noise" of such a service. I have a deep dive on Noise Reduction you can read, but in short, the options include:

• Setting a schedule so that Renovate only creates PRs during certain hours/days
• Grouping together related packages to reduce the total count of PRs
• Automerging "safe" updates, to reduce the need for human intervention

Install Renovate

Installing Renovate is as simple as going to https://github.com/apps/renovate and installing it for the Webpack organisation.

Once you do this, you will first receive an "onboarding" PR per-repository which suggests to you the default config and previews what PRs you will receive once the onboarding PR is merged. You can update the renovate.json file within this onboarding PR to customise behaviour and the App will then update the preview/description each time until you're satisfied.

The two major things I think you will need to decide are (a) which branch to target, and (b) version pinning strategy.

I see that currently you are using next as your branch for 4.x development, while master is maintained for 3.x. Renovate can be configured to target a non-master base branch (e.g. next) although I'm wondering if actually you might want both maintained in parallel.

I recommend pinning all dependencies for maximum build reproducibility and reliability. Although your lockfile helps with this, it doesn't solve every problem - see my article Dependency Pinning for details on that. Of course, you can configure it however you want.

I've run Renovate against a copy of the webpack repository so you can see what that Onboarding would look like in advance: renovate-tests/webpack#1

So far the config is pretty simple:

{
  "extends": [
    "config:js-app"
  ],
  "ignorePaths": ["test/"]
}

Once you've closed the Onboarding PR, Renovate will begin submitting PRs with package updates.

[michael-ciniawsky]

I don't think this is needed and automatic dependency management (anykind) adds a lot of noise to repos. I'm -1 on this change atm

cc @evilebottnawi