Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 9 additions & 54 deletions .env.testing
Original file line number Diff line number Diff line change
@@ -1,63 +1,18 @@
APP_NAME="InvoicePlane v2"
APP_ENV=testing
APP_KEY=base64:JdgrYNc+daEj95jsjJIsYH2/wudsvwvi9LhR1QzFy08=
APP_DEBUG=false
APP_EXTREME_LOGGING=false
DEBUGBAR_ENABLED=false
APP_URL=http://localhost

APP_LOCALE=en
APP_FALLBACK_LOCALE=en
APP_FAKER_LOCALE=en_US

APP_MAINTENANCE_DRIVER=file

PHP_CLI_SERVER_WORKERS=4

BCRYPT_ROUNDS=4
CACHE_STORE=array
IMPORT_DB_DATABASE=invoiceplane_test
MAIL_MAILER=array
PULSE_ENABLED=false
QUEUE_CONNECTION=sync
SESSION_DRIVER=array
TELESCOPE_ENABLED=false

LOG_CHANNEL=stack
LOG_DAILY_DAYS=7
LOG_DEPRECATIONS_CHANNEL=null
LOG_LEVEL=debug

DB_CONNECTION=sqlite
DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=:memory:
DB_DATABASE=invoiceplane_test
DB_USERNAME=root
DB_PASSWORD=root

SESSION_DRIVER=array
SESSION_LIFETIME=120
SESSION_ENCRYPT=false
SESSION_PATH=/
SESSION_DOMAIN=null

BROADCAST_CONNECTION=log
FILESYSTEM_DISK=local
QUEUE_CONNECTION=sync

CACHE_STORE=array

REDIS_CLIENT=phpredis
REDIS_HOST=127.0.0.1
REDIS_PASSWORD=null
REDIS_PORT=6379

MAIL_MAILER=array
MAIL_SCHEME=null
MAIL_HOST=127.0.0.1
MAIL_PORT=2525
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_FROM_ADDRESS="hello@example.com"
MAIL_FROM_NAME="${APP_NAME}"

AWS_ACCESS_KEY_ID=
AWS_SECRET_ACCESS_KEY=
AWS_DEFAULT_REGION=us-east-1
AWS_BUCKET=
AWS_USE_PATH_STYLE_ENDPOINT=false

VITE_APP_NAME="${APP_NAME}"
31 changes: 24 additions & 7 deletions .github/DOCKER.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,21 +44,38 @@ Visit: http://localhost:8080 (override the port with `APP_PORT` in `.env`).

Both PHP images ship the full extension set the app needs: `intl`, `gd`,
`pdo_mysql`, `bcmath`, `zip`, `exif`, `soap`, `redis`. The CLI image also has
Composer, a 1G memory limit for the test suite, and bundled `pdo_sqlite`
(the suite runs on an in-memory sqlite database — no db service needed for
tests).
Composer and a 1G memory limit for the test suite.

---

## Running the test suite

```bash
docker compose run --rm cli vendor/bin/phpunit --exclude-group failing,troubleshooting
docker compose run --rm cli php artisan test --exclude-group failing,troubleshooting
```

`APP_ENV=testing` is the `cli` service default, so `.env.testing`
(sqlite `:memory:`) is picked up automatically. See `RUNNING_TESTS.md` for
filters, groups, and suites.
Use `php artisan test`, not `vendor/bin/phpunit` directly — the two have been observed to behave
differently for this app: a raw `vendor/bin/phpunit` run silently drops some submitted field
values in Livewire form tests. `artisan test` is the proven-reliable path and is what CI uses, so
standardize on it.

**Known issue (see [#689](https://github.com/InvoicePlane/InvoicePlane-v2/issues/689)):** a
freshly-`docker compose build`'t `cli` image has, at least once, reproduced this same
field-dropping bug at scale (100+ false failures) even under `artisan test`, for reasons not yet
isolated — despite extension/ini parity with a known-good image. Before trusting a full local run
from a rebuilt `cli` image, sanity-check it against a small, known test first, e.g.:
```bash
docker compose run --rm cli php artisan test --filter=ContactsTest
```
All 11 assertions should pass. If any fail with "field is required" errors on data you know you
supplied, don't trust the rest of that run — see the linked issue.

`APP_ENV=testing` is the `cli` service default, and it always connects to
the compose stack's real `db` service (MariaDB) for tests — the `cli`
service injects `DB_CONNECTION=mysql`/`DB_HOST=db`/etc. itself, so nothing
in `.env.testing` needs editing. This intentionally does not fall back to
SQLite: SQLite's lenient identifier quoting has masked real bugs before that
only surfaced against MariaDB in CI.

### File ownership on Linux

Expand Down
196 changes: 119 additions & 77 deletions .github/IMPORTING.md
Original file line number Diff line number Diff line change
@@ -1,101 +1,143 @@
# Importing Data
# Data Import & Export Guide

InvoicePlane supports importing data from external systems using CSV files. This guide outlines the requirements and steps for successful data import.
InvoicePlane v2 supports importing data from InvoicePlane v1 databases and exporting data to CSV/Excel formats. This guide covers both workflows.

---

## 📂 Accessing the Import Tool
## Exporting Data

1. Navigate to **Settings**.
2. Click on **Import Data**.
All modules support exporting data in CSV and Excel formats:

- **Clients & Contacts**: Export all relation and contact records
- **Invoices & Quotes**: Export with full line items and totals
- **Payments**: Export payment history with invoice links
- **Products & Categories**: Export product catalog with pricing
- **Projects & Tasks**: Export projects and associated tasks
- **Expenses**: Export expense records with categories
- **And more**: Consistent export support across all modules

### How to Export

1. Navigate to any list page (e.g., Invoices, Clients, Products)
2. Use the **Export** action (typically in the header or row actions)
3. Choose format: **CSV** or **Excel**
4. Two versions available:
- **v2 Format**: Native InvoicePlane v2 schema
- **v1-Legacy Format**: Compatible with InvoicePlane v1 for backward compatibility

---

## 📄 Import Requirements
## Importing from InvoicePlane v1

To ensure a successful import:
The `import:db` command provides a robust pathway to migrate data from InvoicePlane v1 installations.

- **File Format**: Files must be in **comma-delimited CSV** format.
- **File Names**: Use the exact file names as listed below.
- **Headers**: The first row must contain headers matching the specified column names.
- **Columns**: All required columns must be present, even if some fields are empty.
- **File Location**: Place CSV files in the `uploads/import` directory of your InvoicePlane installation.
- **User Email**: The `user_email` in `invoices.csv` must correspond to an existing user in InvoicePlane.
### Requirements

*Note: Failure to meet these requirements may result in import errors.*
- **Source**: SQL dump from a v1 installation (e.g., `backup.sql`)
- **Location**: Place dump file in `storage/app/private/imports/`
- **Supported Entities**: 15 entity types with full data integrity:
- Tax Rates, Products & Categories, Custom Fields
- Users, Clients & Contacts
- Invoice Groups (Numbering), Invoices & Items
- Quotes & Items
- Payments
- Projects & Tasks
- Recurring Invoices
- Uploads & Attachments
- Email Templates, Settings, Notes

---
### Quick Start

#### Dry Run (Preview without Importing)

```bash
php artisan import:db backup.sql --dry-run
```

This shows:
- How many records exist in the source
- How many will migrate successfully
- Which records cannot be imported and why
- A detailed notes section explaining skipped data

#### Import into New Company

```bash
php artisan import:db backup.sql
```

Creates a new company named `{filename} - {YYYY-MM-DD HH:MM:SS}` and imports all compatible records.

#### Import into Existing Company

```bash
php artisan import:db backup.sql --company_id=22
```

If the company doesn't exist, it will be created with that ID.

## 📁 Supported Files and Structures

### 1. `customers.csv`

| Column Name | Description |
|---------------------|-------------------------------------|
| `client_name` | Customer's full name |
| `client_address_1` | Primary address line |
| `client_address_2` | Secondary address line |
| `client_city` | City |
| `client_state` | State or province |
| `client_zip` | ZIP or postal code |
| `client_country` | Country |
| `client_phone` | Phone number |
| `client_fax` | Fax number |
| `client_mobile` | Mobile number |
| `client_email` | Email address |
| `client_web` | Website URL |
| `client_vat_id` | VAT identification number |
| `client_tax_code` | Tax code |
| `client_active` | Status (`1` for active, `0` for inactive) |

### 2. `invoices.csv`

| Column Name | Description |
|-------------------------|-------------------------------------------|
| `user_email` | Email of the InvoicePlane user |
| `client_name` | Name of the customer |
| `invoice_date_created` | Creation date (`YYYY-MM-DD`) |
| `invoice_date_due` | Due date (`YYYY-MM-DD`) |
| `invoice_number` | Unique invoice number |
| `invoice_terms` | Payment terms |

### 3. `invoice_items.csv`

| Column Name | Description |
|--------------------|-------------------------------------------|
| `invoice_number` | Associated invoice number |
| `item_tax_rate` | Tax rate (e.g., `7.8` for 7.8%) |
| `item_date_added` | Date added (`YYYY-MM-DD`) |
| `item_name` | Name of the item |
| `item_description` | Description of the item |
| `item_quantity` | Quantity of the item |
| `item_price` | Price per item (numeric, no currency symbols) |

### 4. `payments.csv`

| Column Name | Description |
|------------------|-------------------------------------------|
| `invoice_number` | Associated invoice number |
| `payment_method` | Method of payment (e.g., Cash, Credit) |
| `payment_date` | Date of payment (`YYYY-MM-DD`) |
| `payment_amount` | Amount paid (numeric, no currency symbols)|
| `payment_note` | Additional notes |
### Features

- **Idempotent**: Re-running the same import twice skips already-imported records
- **Dry Run Support**: Preview results before committing
- **Financial Reconciliation**: Validates that totals in invoices/quotes match their line items
- **Rollback Capable**: Store the batch ID to rollback if needed (future feature)
- **Error Resilience**: Handles real-world data quality issues (missing fields, oversized values, orphaned records)

### Understanding the Output

After import, you'll see:

```
Migration Results:
+-----------+----------+---------+--------+
| Entity | Migrated | Skipped | Errors |
+-----------+----------+---------+--------+
| Invoices | 1,623 | 0 | 0 |
| Clients | 890 | 0 | 0 |
| Payments | 748 | 0 | 0 |
...
```

**Skipped records** are documented in the Details section, with reasons (e.g., "Product row #363 has empty name, will be skipped").

### Known Limitations

1. **Email Templates**: The v2 `EmailTemplateType` enum is misconfigured; email templates may not import correctly (workaround: manually recreate in v2)
2. **File Attachments**: File contents are not included in SQL dumps; re-upload manually
3. **User Passwords**: v1 password hashes are not compatible; users must reset passwords or use SSO

---

## ⚠️ Important Notes
## Future: CSV & ImportAction UI

- **Custom Fields**: Importing custom fields is not supported in the current version.
- **Data Validation**: Ensure all data is accurate and conforms to the required formats to prevent import errors.
- **Testing**: It's recommended to test imports with a small dataset before full-scale importing.
The following import methods are planned but not yet implemented:

- **CSV Import UI**: Per-module import wizards via Filament `ImportAction`
- **Excel Import**: Read Excel files directly
- **Bulk Operations**: Import products from external catalogs, clients from spreadsheets

Track progress on [issue #85](https://github.com/InvoicePlane/InvoicePlane-v2/issues/85).

---

## 🛠️ Troubleshooting
## Troubleshooting

### Import Errors

- **Import Errors**: If the import process fails, double-check file formats, headers, and data consistency.
- **Community Support**: For assistance, visit the [InvoicePlane Community Forums](https://community.invoiceplane.com/).
- **"Dump file not found"**: Ensure the SQL file is in `storage/app/private/imports/`
- **"No company found for your account"**: The system user must be attached to a company before import
- **"Connection failed"**: v1 database credentials may be wrong (only for direct DB imports, not SQL dumps)

### Validation Errors

- **Email validation fails**: Check that email addresses in v1 are valid format
- **Invoice totals mismatch**: Line item amounts don't sum to invoice total; review in v1 before importing
- **Missing foreign keys**: Clients must exist before invoices can reference them

---

*For more information and updates, refer to the [InvoicePlane Wiki](https://wiki.invoiceplane.com/en/2.0/system/importing-data).*
## Support

- **Community**: [InvoicePlane Community Forums](https://community.invoiceplane.com/)
- **Docs**: [InvoicePlane Wiki](https://wiki.invoiceplane.com/)
10 changes: 4 additions & 6 deletions .github/workflows/phpunit.yml
Original file line number Diff line number Diff line change
Expand Up @@ -51,9 +51,7 @@ jobs:
run: php artisan migrate --force --env=testing

- name: Run PHPUnit
# No --exclude-group flag here on purpose: passing it explicitly on the
# CLI was found to override (not add to) phpunit.xml's own <groups>
# <exclude> config, causing failing/flaky/troubleshooting-tagged tests
# to run anyway — confirmed by testing both ways. phpunit.xml's own
# config already excludes them; rely on that instead.
run: php artisan test --env=testing
# Matches the Makefile's local-dev default (see Makefile's _phpunit/_artisan
# vars): failing/flaky/troubleshooting-tagged tests are known issues tracked
# separately, not blockers for this run.
run: php artisan test --env=testing --exclude-group failing,flaky,troubleshooting
5 changes: 2 additions & 3 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,10 +10,9 @@ Laravel 11 + Filament v4 + Livewire v3 invoicing app. Modular architecture via `
composer install
cp .env.example .env && php artisan key:generate
php artisan migrate && php artisan db:seed
# Tests (no MySQL locally? use SQLite)
# Tests run against real MariaDB — no SQLite fallback (parity with CI)
cp .env.testing.example .env.testing
# set DB_CONNECTION=sqlite, DB_DATABASE=:memory: in .env.testing
php artisan test
docker compose run --rm cli php artisan test --exclude-group failing,troubleshooting
```

---
Expand Down
16 changes: 13 additions & 3 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -195,11 +195,21 @@ User::factory()->create(['is_active' => true, 'email_verified_at' => now()])

### DB for tests

Tests need a DB. Production CI uses MariaDB 11. For local dev without MySQL, set in `.env.testing`:
Tests need a real MariaDB DB — matching CI (MariaDB 11) — not SQLite. SQLite's lenient identifier
quoting has silently masked real bugs before (e.g. `->latest()` defaulting to a nonexistent
`created_at` column on `$timestamps = false` models passed locally, failed on CI). Run via the
`cli` compose service, which points at the stack's `db` service automatically:
```
DB_CONNECTION=sqlite
DB_DATABASE=:memory:
docker compose run --rm cli php artisan test --exclude-group failing,troubleshooting
```
No `.env.testing` edits needed — the `cli` service injects `DB_CONNECTION=mysql`/`DB_HOST=db` etc.
itself. Use `php artisan test`, not `vendor/bin/phpunit` directly — the two have been observed to
behave differently for this app's Livewire form tests; `artisan test` is the reliable one.
**Known issue:** a freshly-rebuilt `cli` image has reproduced false Livewire-form failures at
scale even under `artisan test`, for reasons not yet isolated — see
[#689](https://github.com/InvoicePlane/InvoicePlane-v2/issues/689) and sanity-check with
`--filter=ContactsTest` (should be 11/11 passing) before trusting a full run from a rebuilt image.
See `.github/DOCKER.md`.

### AAA phase comment style

Expand Down
Loading
Loading