From c9fb537fe301a0755f3716d121b10e85d0239d9b Mon Sep 17 00:00:00 2001 From: Alex Cronin Date: Tue, 19 Aug 2025 10:07:26 +0100 Subject: [PATCH] ReadMe Updated for additional structure, known issues and Visual C++ Redistributable dependancy. --- Notes.txt | 12 +++- README.md | 172 +++++++++++++++++++++++++++++++++++++----------------- 2 files changed, 130 insertions(+), 54 deletions(-) diff --git a/Notes.txt b/Notes.txt index a659bcf..bac66f1 100644 --- a/Notes.txt +++ b/Notes.txt @@ -14,4 +14,14 @@ Images - reduce the size of the `load your key into secrets` image as it take a large portion of a page This sentence is unclear -- But you don't consider getting your brain to learn the difference between a good/bad/urgent/valuable invoice/request/email/sales lead training a model. \ No newline at end of file +- But you don't consider getting your brain to learn the difference between a good/bad/urgent/valuable invoice/request/email/sales lead training a model. + + +# Added 2025.08.19 +Training Times +Linux +- Dell i7 64GB no GPU training time = 38 mins +- MacBook Pro M1 training time = 13 mins + +ReadMe +- Updated in detail, please review and verify it is correct. \ No newline at end of file diff --git a/README.md b/README.md index 4726dea..17baa18 100644 --- a/README.md +++ b/README.md @@ -18,90 +18,135 @@ This project shows you how to take an idea ("I want to classify emails automatic ### 1. Clone and Setup +#### 1.1 Mac OSX / Linux -In Mac osx -```bash - -# 1) Install uv (one time). If you already have python you are happy with skip this step and make a folder -# macOS/Linux: -#curl -LsSf https://astral.sh/uv/install.sh | sh - +##### 1.1.1 Install uv (one time) +If you already have python you are happy with skip this step and make a folder +``` +# curl -LsSf https://astral.sh/uv/install.sh | sh +uv --version +``` +##### 1.2.2 Download project repository +``` +git --version git clone https://github.com/cavedave/ai-email-classifier.git -# 2) Project folder +``` +Change directory in to the downloaded repo to verify it has succeeded +``` cd ai-email-classifier +ls +``` +###### 1.2.3 Create and activate virtual environment +``` uv venv --python 3.13 venv -# 3) Activate virtual environment source venv/bin/activate # On Windows: venv\Scripts\activate +``` +##### 1.2.4 GPU Set Up +Not typically required - -# 3) Create env + install deps (very fast) +##### 1.2.4 Install deps (very fast) +``` uv pip install -U pip uv pip install jupyterlab ipykernel pandas scikit-learn matplotlib tqdm \ transformers accelerate huggingface_hub \ torch \ google-genai ipywidgets seaborn datasets - - - ``` -in windows is a bit of work - -First off find what GPU you have. -```bash - -# 0) (One time) Install uv -irm https://astral.sh/uv/install.ps1 | iex +#### 1.2 Windows +In windows is a bit of work +If you may need to install https://aka.ms/vs/17/release/vc_redist.x64.exe from +https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170 +The following command should be executed in powershell + +##### 1.2.1 Install uv (one time) +If you already have python you are happy with skip this step and make a folder +``` +# irm https://astral.sh/uv/install.ps1 | iex +uv --version +``` Find where uv is installed and write it down -# Find uv.exe +Find uv.exe + +``` Get-ChildItem -Recurse $env:USERPROFILE -Filter uv.exe -ErrorAction SilentlyContinue +``` +Expected output:`C:\Users\\.local\bin` -this will output something like -$uvBin = 'C:\Users\reall\.local\bin' -#double check its thereß +Create a variable to reference location +``` +$uvBin = 'C:\Users\\.local\bin' ` +``` +Double check `uv` is there +``` if (-not (Test-Path "$uvBin\uv.exe")) { Write-Error "uv.exe not found in $uvBin"; exit 1 } - -# Add to PATH for current session if missing +``` +Add `uv` to PATH for current session if missing +``` if (-not ($env:Path -split ';' | Where-Object { $_ -ieq $uvBin })) { $env:Path = "$uvBin;$env:Path" } +``` +Verify that the $uvBin path is in your system path +``` +echo $env:Path +``` +Open a new terminal and ensure `uv` can be found +```dotnetcli +uv --version +``` -# 1) Get the project +##### 1.2.2 Download project repository + +``` +git --version git clone https://github.com/cavedave/ai-email-classifier.git -cd ai-email-classifier +``` -##if git doesnt work +If if `git` is not on your machine you can get the project using `curl` +Option 1 - use curl & tar +```dotnetcli +C:\Windows\System32\curl.exe -L -o ai-email-classifier.zip https://github.com/cavedave/ai-email-classifier/archive/refs/heads/main.zip +tar -xf tar -xf ai-email-classifier.zip +``` +Option 2 - use the default windows options +``` +Invoke-WebRequest -Uri https://github.com/cavedave/ai-email-classifier/archive/refs/heads/main.zip -OutFile ai-email-classifier.zip +Expand-Archive -Path ai-email-classifier.zip -DestinationPath . +``` -# Verify -git --version +Change directory in to the downloaded repo to verify it has succeeded +``` +cd ai-email-classifier-main +ls +``` -# 2) Create & activate a Python 3.12 virtual env +##### 1.2.3 Create and activate virtual environment +``` uv venv --python 3.12 venv .\venv\Scripts\Activate.ps1 +``` +##### 1.2.4 GPU Set Up -Find your driver as in cuda126 is assume dbelow -# Pick ONE that matches your driver (example uses CUDA 12.6): +Find what GPU you have. +```bash +wmic path win32_videocontroller get name +``` +Find your driver as in cuda126 is assumed below +Pick ONE that matches your driver (example uses CUDA 12.6): +``` uv pip install --index-url https://download.pytorch.org/whl/cu126 torch torchvision torchaudio -# 3) Install deps (CPU) -uv pip install -U pip -uv pip install jupyterlab ipykernel pandas scikit-learn matplotlib tqdm transformers accelerate huggingface_hub torch google-genai ipywidgets seaborn datasets - - - ``` -If you dont have git download the source this way -```bash - -curl -L -o ai-email-classifier.zip \ - https://github.com/cavedave/ai-email-classifier/archive/refs/heads/main.zip -unzip ai-email-classifier.zip -cd ai-email-classifier-main +##### 1.2.5 Install deps (CPU) +``` +uv pip install -U pip +uv pip install jupyterlab ipykernel pandas scikit-learn matplotlib tqdm transformers accelerate huggingface_hub torch google-genai ipywidgets seaborn datasets ``` ### 2. Launch Jupyter Lab @@ -127,14 +172,11 @@ Navigate to `notebooks/train_bert_model_CLEAN.ipynb` and run the cells to: ## Learning Path - ## Key Technologies - **Python +** - **Transformers (Hugging Face)**: BERT model implementation - **PyTorch**: Deep learning framework - - - **Pandas**: Data manipulation @@ -162,8 +204,32 @@ MIT License - feel free to use this for your own projects! ## Next Steps Ready to build your own AI tool? Start with: -1. **Understand the architecture** in `docs/` -2. **Follow the training tutorial** in the notebook +1. **Understand the architecture** in `docs/` +2. **Follow the training tutorial** in the notebookjupyter lab 3. **Customize for your domain** 4. **Deploy and iterate!** +# Known Issues + +## Windows install + +### Error 1 +```dotnetcli +... +OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\vboxuser\Desktop\ai-email-classifier-main\venv\Lib\site-packages\torch\lib\c10.dll" or one of its dependencies. +``` +#### Solution: +Install Microsoft Visual C++ Redistributable by the link provided at the top of the error, The link below provides more information +Install https://learn.microsoft.com/en-us/cpp/windows/latest-supported-vc-redist?view=msvc-170 + +### Error 2 +```dotnetcli +... +ImportError: cannot import name 'GenerationMixin' from 'transformers.generation' (C:\Users\vboxuser\Desktop\ai-email-classifier-main\venv\Lib\site-packages\transformers\generation\__init__.py) +``` +##### Solution +```dotnetcli +pip uninstall transformers torch +pip install transformers torch +``` +Alternatively delete and recreate virtual environment