Truffle speed improvements; remove extra compilation from tests

[LogvinovLeon]

• I've asked for help in the Truffle Gitter before filing this issue.

---

Issue

Truffle could be faster. We need to profile it, find bottlenecks and implement some caching.

• truffle version takes 4s on my computer
• truffle test with solidity tests doesn't cache compiled contracts and artifacts and rebuilds them on every run

Steps to Reproduce

• run truffle version
• run truffle tests with solidity tests

Expected Behavior

• I expect time for truffle version to be negligible
• I expect second test run to be faster than the first one and use precompiled contracts or only recompile the changed ones.
  • Maybe use a temp directory
  • Maybe have a directory inside of a project ignored in a generated .gitignore

Actual Results

Above^

Environment

• Operating System: macos
• Hardware: Macbook Retina 12 inch early 2016 1.1GHz Intel core m3
• Truffle version: 3.0.5
• Ethereum client: testrpc
• node version: v7.5.0
• npm version: 4.1.2

[LogvinovLeon]

Talking about the first part of this issue: (Every truffle command takes at least 4 seconds)

I profiled the code and ~60% of time is spent in a C++ function which parses the module.
Here is the profiler output
It means, that the dependency tree is too big and it takes a lot of time for node to load all the modules.

Not all this code is required every time we run truffle, so the solution would be to make some of requires lazy. This will cut huge parts of dependency tree and load them on-demand.
Node made the decision, that require should not be lazy, because it was designed for writing web-serers, where startup-time is less important than latency, but if you use require inside of a function it is not executed before you actually call the function.

I propose to split by command, cause it's the most natural way to do it. We don't need to load a compiler dependencies on truffe version run.

I already tested that approach on my computer and it speeds up a lot.
For example the execution time for truffle version:

• Before

Truffle v3.0.4

real	0m4.759s
user	0m4.112s
sys	0m0.337s

• After

Truffle v3.0.4

real	0m0.439s
user	0m0.404s
sys	0m0.055s

Here is my diff
It just moves required inside of a run function for commands.
If you hae better ideas, I'm happy to discuss them.

I had an idea of making the file /lib/commands/index.js the splitpoint by replacing for example

compile: require("./compile"),

with

compile: () => require("./compile"),

but this is currently not possible, cause we need all the command info for yargs.

[LogvinovLeon]

Talking about the second part of this issue: (Contracts are recompiled every time we run the tests)

I wrote an example implementation that caches the precompiled contracts in tests.

It's not perfect and it's a subject to discussion, but I personally am happy with that and I'm getting faster test runs.

First I used temp directory compiled_test_contracts to store compiled contracts.
The directory is in system-specific location outside of the project folder.
After implementing that I experienced an error, because Contracts.compile didn't return the paths. It has a callback with three arguments (error, abstractions, paths).
It's because in truffle-compile/index.js:118 we call the same callback with no arguments.
I tried to call that callback with an empty array of paths, but had a problem, that a second test run failed, because the dependencies were not linked.

So I just added some code for getting required deps, that does not require on paths and it started to work.

I understand, that that is not a perfect solution and I'm opened for a discussion.

[tcoulter]

#355 has been merged.

Still looking into your solidity-test compile speed up, but it will come soon. Thanks again!

[zie1ony]

It takes up to 2s to load solc module. I made small PR: ConsenSys-archive/truffle-compile#19 to fix it.

[zie1ony]

In order not to compile every time you test, make sure your build/contracts is up to date. If it's not, then truffle test will compile changed *.sol files to have latest version in /tmp/test-....

Take a look:

/code # truffle compile
Compiling ./contracts/TokenMarket.sol...
Writing artifacts to ./build/contracts

/code # truffle compile
/code # truffle test
Using network 'development'.

  Contract: TokenMarket
    ✓ should allow to check day of sale (861ms)
  1 passing (2s)

/code # echo " " >> contracts/TokenMarket.sol
/code # ./truffle test
Using network 'development'.

Compiling ./contracts/TokenMarket.sol...

  Contract: TokenMarket
    ✓ should allow to check day of sale (931ms)
  1 passing (2s)

/code # ./truffle test
Using network 'development'.

Compiling ./contracts/TokenMarket.sol...

  Contract: TokenMarket
    ✓ should allow to check day of sale (713ms)
  1 passing (2s)

/code # ./truffle compile
Compiling ./contracts/TokenMarket.sol...
Writing artifacts to ./build/contracts

/code # ./truffle test
Using network 'development'.

  Contract: TokenMarket
    ✓ should allow to check day of sale (962ms)
  1 passing (2s)

/code # 

So best way to avoid unnecessary compilation is test with: truffle compile && truffle test.

Hope that helps.

[troggy]

truffle compile && truffle test is not working for me. It still recompiles everything. For now, I opted to the workaround by @LogvinovLeon