feat: package manager support by 0618 · Pull Request #960 · aws-amplify/amplify-backend

0618 · 2024-01-30T22:57:27Z

Problem

Branch poc/package-manager-support has the following major changes:

add a 'poc-e2e-flow-test' to run package_manager_sanity_checks.test.ts check against supported Package Managers
refactored npmPackageManagerController to packageManagerContollerFactory and packageManagerController. And moved them to cli-core
refactored backed-deploy to use packageManagerContollerFactory

Related PR:

TODO:

Issue number, if available:

Changes

Corresponding docs PR, if applicable:

Validation

Checklist

If this PR includes a functional change to the runtime behavior of the code, I have added or updated automated test coverage for this change.
If this PR requires a change to the Project Architecture README, I have included that update in this PR.
If this PR requires a docs update, I have linked to that docs PR above.
If this PR modifies E2E tests, makes changes to resource provisioning, or makes SDK calls, I have run the PR checks with the run-e2e label set.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

* feat: add env ci support * test: update unit tests * feat: create amplify --yes option * chore: remove un-used code * feat: support Panage Manager env var * chore: add comments * chore: remove env.CI * chore: update changeset * chore: rename PACKAGE_MANAGER_EXECUTABLE * fix: tests * fix npm -> npx

* test: init e2e flow test * fix GH hash * use dynamic pkg manager * setup nodejs with npm * setup more pkg managers * install pkg managers * fix yarn1 and change step name * fix yarn1 * fix yarn1 * split yarn1 into 2 steps * fix yarn1 * fix yarn1 win * exclude bun on windows * yarn1 windows * setup yarn3 * yarn3 * yarn3 yarnPath * yarn use 3rd party action * try yarn3 * set yarn 3.6.x * set yarn berry * set yarn stable and pass --yes * feat: add env ci support * test: update unit tests * feat: create amplify --yes option * chore: remove un-used code * feat: support Panage Manager env var * chore: add comments * chore: remove env.CI * chore: update changeset * chore: rename PACKAGE_MANAGER_EXECUTABLE * fix: tests * use Env Var PACKAGE_MANAGER_EXECUTABLE * fix npm -> npx, remove bun * fix npm -> npx * change -- --yes * chore: update package.lock * chore: remove @Alpha * fix: clean up yarn * fix: change yarn1 to yarn * chore: rename yarn step * chore: use env * fix: yarn-stable env var * add comments * chore: update sample app with new imports (#541) * feat: use "use client"; directive in generated react components (#540) * fix: flatten prop types auth (#534) * fix: flatten auth login with types * chore: update snapshots * chore: update api * chore: add changeset * chore: add TODO --------- Co-authored-by: Amplifiyer <51211245+Amplifiyer@users.noreply.github.com> Co-authored-by: Spencer Stolworthy <sstol@amazon.com> Co-authored-by: awsluja <110861985+awsluja@users.noreply.github.com>

* fix: npm_project_initializer to use env var * fix: pnpm init

* feat: setup GH workflow for create amplify * chore: changeset * fix: add pkg type for modern yarn * fix npm install * feat: test Node v20 and Node v19 * feat: test Node v20 and yarn-stable with Node 19 * feat: exclude yarn-stable * feat: include node 20 and exclude node 19 * feat: use node 20 for others, use node 19 for yarn-stable * chore: add TODO to use Node 20

* chore: change branch to poc/pms-create-amplify * fix: gitIgnore test * fix: refactor e2e for pms * chore: update package.lock * chore: update package.lock * temp: refactor * temp: run 1 test * temp: fix npx * temp: install packages for yarn * temp: update initial_project_file_generator * temp: install ts for yarn * temp: ignore node_modules and yarn.lock * temp: create yarn.lock for yarn stable * temp: fix yarn-stable init * temp: fix --help * fix: assert for gitignore * chore: change yarn stable install * fix: not install yarn stable globally * add @yarnpkg/sdks base * yarn stable use node-modules * remove emoji * remove @yarnpkg/sdks * yarn stable use node 18.18 * yarn stable use node 20 * chore: re-enable all tests * Revert "chore: re-enable all tests" This reverts commit f7a167b. * chore: uncomment test * chore: enable all initialStatues * test: change concurrencyLevel for yarn and yarn stable * add yarn-stable to test fails fast * chore: update changeset * chore: change workflow trigger * chore: yarn not install typescript in root folder * chore: update package.lock * Revert "chore: yarn not install typescript in root folder" This reverts commit 0d83dd0. * chore: remove comment * chore: fix typo * chore: rename yarn-classic and yarn modern

* chore: update package.lock * init * feat: dynamically get package manager * chore: change workflow trigger * chore: change workflow trigger * chore: rename npm functions * chore: update output log * fix: pnpm cache clear command * chore: refactor packageManager * fix: remove pnpm store * chore: create package_manager file * chore: change workflow trigger * chore: refactor PackageManager Controller and Initializer * fix: update types * chore: refactor to use factory * chore: remove try catch from the controllers * chore: change type * chore: use abstract class * fix tests * fix: PackageManagerBase * chore: refactor PackageManagerControllerFactory * chore: refactor packageJsonExists * chore: cleanup code * fix: projectJsonExists * remove package_manager file * chore: refactor package_manager props into package manager controllers * chore: rename packageManagerControllerFactory * chore: make projectRoot, userAgent, getPackageManagerName private * chore: refactor ensureInitialized * chore: refactor PackageManagerController to extend PackageManagerControllerFactory * Revert "chore: refactor PackageManagerController to extend PackageManagerControllerFactory" This reverts commit 7ddc64f. * chore: refactor to inject PackageManagerControllerFactory to xPackageManagerController * chore: move initialProjectFileGenerator to packageManagerController * fix: initializeAMplifyFolder * fix: generateInitialProjectFiles * fix: template path * chore: refactor getPackageManagerController yarn-modern * chore: create getWelcomeMessage * Revert "chore: refactor getPackageManagerController yarn-modern" This reverts commit 28da8a9. * chore: refactor generateInitialProjectFiles for yarn-modern * chore: cleanup code * chore: remove PackageManagerControllerFactory index * chore: remove ensureInitialized * Revert "chore: remove ensureInitialized" This reverts commit 96483ce. * Revert "Revert "chore: remove ensureInitialized"" This reverts commit 7c59c80. * temp: add initializeProject to PackageManagerController * temp: revert InitialProjectFileGenerator * fix: InitialProjectFileGenerator * test: restore initial_project_file_generator * move installDependencies and getWelcomeMessage to PackageManagerController * add addLockFile * add JSDocs and resolve some review comments * fix: create_amplify.test * chore: add addTypescript * chore: refactor generateInitialProjectFiles * Update packages/create-amplify/src/package-manager-controller/package_manager_controller_factory.ts Co-authored-by: Edward Foyle <foyleef@amazon.com> * chore: refactor generateInitialProjectFiles again * chore: refactor welcomeMessage * test: add package_manager_controller_factory * test: fix test types * chore: comments update * chore: handle process.env.npm_config_user_agent undefined * chore: move addLockFile and addTypescript into initializeTsConfig * test: refactor packageManagerControllerFactory * fix: yarn initializeTsConfig * fix: yarn modern initializeTsConfig * fix: amplify_project_creator test * chore: refactor contructor * chore: update package.lock * fix: pnpm init, remove --debug, etc * test: add test for NpmPackageManagerController * fix build error * chore: refactor NpmPackageManagerController * test: add test for xPackageManagerController * chore: convert fs to fsp * fix yarn modern --------- Co-authored-by: Edward Foyle <foyleef@amazon.com>

sobolk · 2024-01-31T18:33:03Z

+      /**
+       * TODO: This is a temporary fix for execa version inconsistency. Remove this once execa version is fixed.
+       * Issue: https://github.com/aws-amplify/amplify-backend/issues/962
+       */
+      if (
+        dependencyVersionUsage.allVersions.size > 1 /* Issue-962 Start */ &&
+        dependencyName !== 'execa' /* Issue-962 End */
+      ) {


as for this one.

we should lift this as an option, so that it can be configured here

amplify-backend/scripts/check_dependencies.ts

Lines 5 to 18 in 9f1a78b

await glob('packages/*'),

{

'aws-amplify': { denyAll: true },

'@aws-amplify/datastore': { denyAll: true },

'@aws-amplify/core': { denyAll: true },

'@aws-amplify/cli-core': {

allowList: [

'@aws-amplify/backend-cli',

'@aws-amplify/sandbox',

'create-amplify',

],

},

},

[['aws-cdk', 'aws-cdk-lib']]

.

Can be a follow up.

…er_controller_factory.ts Co-authored-by: Kamil Sobol <sobkamil@amazon.com>

0618 · 2024-02-01T00:09:47Z

+'@aws-amplify/plugin-types': minor
+'@aws-amplify/cli-core': minor
+'@aws-amplify/sandbox': minor
+'@aws-amplify/backend-cli': minor


are all of the versions minor?

given where we are today (0.x versions and added feature with api breaks) yes that seems ok.

0618 · 2024-02-01T00:11:30Z

+'@aws-amplify/backend-cli': minor
+---
+
+Support more package managers, such as yarn (version 1 and version 2+), pnpm.


Do we also want to add several examples in the changelog?

I'm thinking of adding:

For example the following commands:
yarn create amplify
yarn amplify pipeline-deploy
pnpm create amplify
pnpm amplify pipeline-deploy

yes why not

however if you do you may want to split changeset into two files and add this only for CLI and create amplify

* add package-manager-tests to health-checks * remove poc-e2e-flow-test * add run_package_manager_tests_tests condition * fix run_package_manager_tests_tests condition * remove amplify-backend_windows-latest_8-core * use re-usable action for run-e2e * Revert "remove amplify-backend_windows-latest_8-core" This reverts commit 547834a. * Revert "use re-usable action for run-e2e" This reverts commit 3159e01. * revert re-usable action

sobolk

Looks good.

sobolk · 2024-02-01T14:46:27Z

+      // const writeFileMock = mock.fn(() => Promise.resolve());
+      // mock.method(fsp, 'writeFile', writeFileMock);


please remove commented out code

Done 010277b

Co-authored-by: Kamil Sobol <sobkamil@amazon.com>

sobolk

Ship it!

                                  .                   .
                              _..-''"""\          _.--'"""\
                              |         L         |        \
                  _           / _.-.---.\.        / .-.----.\
                _/-|---     _/<"---'"""""\\`.    /-'--''"""""\
               |       \     |            L`.`-. |            L
               /_.-.---.L    |            \  \  `|            J`.
             _/--'""""  \    F            \L  \  |             L
               L      `. L  J  _.---.-"""-.\`. L_/ _.--|"""""--.\ `.
               |        \+. /-"__.--'""""   \ `./'"---'""""""   \`. `.
               F   _____ \ `F"        `.     \  \                L `.
              /.-'"_|---'"\ |           `    JL |                 L  `.`.
             <-'""         \|    _.-.------._ A J    _.-.-----`.--|   ``.`.
              L         `. |/.-'"_.-`---'""\."| /-'"---'"""""   \`.\.  \ `.`.
              |  _.------\.<'"""            L\ L\                `.`\`. \  `.
         _.-'//'"--'"""   L\|       ________\ `.F     ___.-------._L \ `-\   \`.
        /___| F             F _.--'"_|-------L  /_.-'"_.-|-'"""""""\  L   L   `.`.
            | F  _.-'|"""""/'"-'"""          J <'"""                L J   |     `.`.
            |/-'-''/|""\ )-|\                 F \                   |  L .'"""`\""-\\_
             F`-'-'-(`-')  | \                F  \                  |___`"""`.""`.-'"
------------/        `-'---|  F               L   L             __     |"""""`-'"__________
          .'_         |    |__L          __  J__  |    _.--'""""   `".----'".'
         '""""""""""""|--._+--F _.-'""||"   """___/.-'"   ||-'"/""""" \_. .'
         J------------(___\__/'_____.--------'-------'""""""""           /
         `-.                                        _.__.__.__.____     J_.-._
    .'`-._ (-`--`---.'--._`---._.-'`-._.-'_.-'``-._'               `-''-'

edwardfoyle · 2024-02-01T17:35:09Z

@@ -0,0 +1,11 @@
+---
+'@aws-amplify/integration-tests': minor


nit: this package is private so it doesn't need to be included in changesets

Removed 276b82a

edwardfoyle · 2024-02-01T17:54:30Z

    });

-    await assert.rejects(
+    assert.throws(


I guess if it works it works, but it seems like this should be reverted because executeWithDebugLogger is still returning a promise

I think it works because executeWithDebugLogger return a promise, but itself is not async because it doesn't have the async key word

edwardfoyle · 2024-02-01T17:58:14Z

+  };
+
+  /**
+   * addTypescript - initializes a tsconfig.json file in the project root


nit: either update or remove this jsdoc

Good catch. Done 223f9eb

edwardfoyle · 2024-02-01T18:09:11Z

 */
 export const getProjectRoot = async () => {
-  const useDefault = process.env.npm_config_yes === 'true';
+  const argv = await yargs(process.argv.slice(2)).options({


is this our only usage of yargs in this package? If so, I'd suggest we remove it an just do a check for process.argv.some(arg => arg === '--yes'

I'd rather not.
This will break this:

amplify-backend/packages/integration-tests/src/test-live-dependency-health-checks/health_checks.test.ts

Line 45 in 348717b

await execa('npm', ['create', 'amplify', '--yes', '--', '--help'], {

And. we'll loose --help output :(

having said that, we should probably refactor how do we use yargs in this package.

but then, looking at sources on main I don't see yargs usage in that package's src but I can see yargs in dependencies.
So I wonder how this works now...

bcd0740fa72c:testapp004 sobkamil$ npm create amplify@latest -- --help ? Where should we create your project? (.)

bcd0740fa72c:testapp004 sobkamil$ npm create amplify@0.4.4 -- --help Options: --help Show help [boolean] --version Show version number [boolean] --debug [boolean] [default: false]

Looks like it regressed...

btw. that regression happened on main so if we want to fix it it can be a follow up.

Can't reproduce the regression on amplify@0.6.0, which is this PR ✌️

I'm good with cutting a ticket to track and doing this in a follow up

Issue to remove yargs in a follow-up PR #972

Co-authored-by: Edward Foyle <foyleef@amazon.com>

0618 and others added 30 commits October 30, 2023 11:51

Merge branch 'main' into poc/package-manager-support

b155096

Merge branch 'main' into poc/package-manager-support

3f37820

Merge branch 'main' into poc/package-manager-support

3799a9c

Merge branch 'main' into poc/package-manager-support

3940efa

fix: pnpm init and env var (#563)

6c0036f

* fix: npm_project_initializer to use env var * fix: pnpm init

Merge branch 'main' into poc/package-manager-support

6cb29c6

Merge branch 'main' into poc/package-manager-support

7634c37

Merge branch 'main' into poc/package-manager-support

cb56b3d

Merge branch 'main' into poc/package-manager-support

cbed3f4

chore: update package.lock

6750713

Merge branch 'update-package-lock' into poc/package-manager-support

43d8da3

Merge branch 'main' into poc/package-manager-support

4b30f2a

chore: update package.lock

d5eab4a

Merge branch 'update-package-lock' into poc/package-manager-support

e2bbafc

Merge branch 'main' into poc/package-manager-support

c1edfdf

Merge branch 'main' into poc/package-manager-support

5dbbc62

Merge branch 'main' into poc/package-manager-support

8943ac5

Merge branch 'main' into poc/package-manager-support

ddbc3c5

Merge branch 'main' into poc/package-manager-support

fd3defe

Merge branch 'main' into poc/package-manager-support and use main

6bff8b4

fix(create_amplify.test): after merge main

4c77c70

fix the concurrency for yarn classic and modern, and fix expect

0413a9e

Merge branch 'main' into poc/package-manager-support

af81b8e

Merge branch 'main' into poc/package-manager-support

b6a0050

Merge branch 'main' into poc/package-manager-support

2f51f94

0618 added 4 commits January 30, 2024 21:41

fix getCommand, cdk_deployer.test, amplify_project_creator.test

e6f143e

fix initial_project_file_generator.test

51d6dad

fix executeWithDebugLogger test

99c1b0b

fix YarnModernPackageManagerController test

4bfda61

sobolk reviewed Jan 31, 2024

View reviewed changes

0618 and others added 5 commits January 31, 2024 10:47

fix packageManagerControllerFactory test

4ee0cf2

fix lint error

ead5864

Update packages/cli-core/src/package-manager-controller/package_manag…

c4a03ad

…er_controller_factory.ts Co-authored-by: Kamil Sobol <sobkamil@amazon.com>

remove never used packageManagerExecutable

e344b5b

update changeset

a4011c4

0618 commented Feb 1, 2024

View reviewed changes

0618 added the run-e2e Label that will include e2e tests in PR checks workflow label Feb 1, 2024

sobolk reviewed Feb 1, 2024

View reviewed changes

Comment thread .changeset/new-timers-warn.md Outdated

sobolk previously approved these changes Feb 1, 2024

View reviewed changes

Update .changeset/new-timers-warn.md

1f297f0

Co-authored-by: Kamil Sobol <sobkamil@amazon.com>

0618 dismissed sobolk’s stale review via 1f297f0 February 1, 2024 15:33

0618 added 2 commits February 1, 2024 07:34

clean up commented code

010277b

use amplify-backend_windows-latest_8-core

4f85b3d

sobolk previously approved these changes Feb 1, 2024

View reviewed changes

edwardfoyle reviewed Feb 1, 2024

View reviewed changes

Update .changeset/new-timers-warn.md

e98d642

Co-authored-by: Edward Foyle <foyleef@amazon.com>

0618 dismissed sobolk’s stale review via e98d642 February 1, 2024 19:11

0618 added 2 commits February 1, 2024 11:20

remove inaccurate JSdoc

223f9eb

remove integration-tests from changeset

276b82a

edwardfoyle approved these changes Feb 1, 2024

View reviewed changes

0618 mentioned this pull request Feb 1, 2024

remove yargs from create-amplify #972

Open

sobolk approved these changes Feb 1, 2024

View reviewed changes

sobolk merged commit b73d76a into main Feb 1, 2024

sobolk deleted the poc/package-manager-support branch February 1, 2024 20:00

	await glob('packages/*'),
	{
	'aws-amplify': { denyAll: true },
	'@aws-amplify/datastore': { denyAll: true },
	'@aws-amplify/core': { denyAll: true },
	'@aws-amplify/cli-core': {
	allowList: [
	'@aws-amplify/backend-cli',
	'@aws-amplify/sandbox',
	'create-amplify',
	],
	},
	},
	[['aws-cdk', 'aws-cdk-lib']]

		// const writeFileMock = mock.fn(() => Promise.resolve());
		// mock.method(fsp, 'writeFile', writeFileMock);

Conversation

0618 commented Jan 30, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Problem

Changes

Validation

Checklist

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

sobolk left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sobolk left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

0618 Feb 1, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

0618 commented Jan 30, 2024 •

edited

Loading

0618 Feb 1, 2024 •

edited

Loading