Create exception handling to skip nfs created files causing errors in spack manager distribution create mirror

[akimler]

We are running into problems using NFS during the spack mirror distribution --source-only command. This issue happens after we create the source mirror for the new environment when we try to remove the directories in the new environment:

Example:

$ spack manager distribution --source-only --distro-dir $COMMON_SOURCE_PACKAGE_DIR --extra-data extra_data --exclude-specs cth,acis --exclude-configs "config:build_stage,config:install_tree:root"
==> Concretizing env: /rnfs01/sierra/builds/sVRHJykUM/001/1540-compsim/sierra/automation/install-env....
==> Precleaning....
==> Excluding configurations: ['config:build_stage', 'config:install_tree:root']....
==> Executing: spack config remove config:build_stage
==> Executing: spack config remove config:install_tree:root
==> Adding specs to env: /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/environment....
...
...
Executing: spack mirror create --directory /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/source-mirror --exclude-specs cth acis sierra base adagio acme ...
...
...
==> Adding package zuzax@master.latest to mirror
==> Adding package zuzax-apps@master.latest to mirror
==> Error: [Errno 39] Directory not empty: '/rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/environment/.spack-env'
==> Archive stats:
    117  already present
    77   added
    0    failed to fetch.
==> Adding mirror to env: /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/environment....
==> Executing: spack mirror add internal-source ../source-mirror
==> Packing up Spack installation to /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/spack....
==> Packaging up extra data to /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package....
==> /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/spack/var/spack
==> /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/spack/.git-blame-ignore-revs
==> /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/spack/.git
==> /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/spack/.gitattributes
==> /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/spack/.gitignore
==> /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/spack/etc/spack/README.md
==> /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/spack/etc/spack/packages.yaml
==> /rnfs01/sierra/.ci_pipelines/5496552/cts1+_source_package/spack/etc/spack/site
Slurm job ci-38416033_1783296880 state: FAILED

We need a way to gracefully handle the error above in distribution.py.