diff --git a/CHANGELOG.md b/CHANGELOG.md index f1c1728..9a48d55 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,22 @@ All notable changes to `scanii-lambda` are documented here. Versions follow [SemVer](https://semver.org). +## [3.0.2] — Deployment README refresh + +### Changed + +- Rewrote the README around the AWS Serverless Application Repository customer + deployment flow, including parameters, required post-deploy S3 trigger setup, + result actions, and deployment verification. +- Moved the AWS deployment link into a prominent `Deploy on AWS` section and + moved brittle endpoint enumeration out of the README in favor of the + maintained endpoint documentation. +- Documented the advanced environment configuration defaults inline in + `lib/config.js`. +- Corrected the SAM parameter defaults for `actionTagObject` and + `actionDeleteObjectOnFinding` to use the documented and accepted `true` / + `false` values. + ## [3.0.1] — Node.js 24 handler compatibility ### Fixed diff --git a/README.md b/README.md index 8ac338d..4327a4d 100644 --- a/README.md +++ b/README.md @@ -1,34 +1,156 @@ -# Uva Software’s scanii-lambda -A Sam-Packaged AWS Lambda client to the [scanii.com](https://scanii.com) content processing engine. For a detailed walk-through of deploying this application see: https://docs.scanii.com/article/151-how-do-i-analyze-content-stored-on-amazon-s3. +# Scanii Lambda for Amazon S3 -## How it works -This is, essentially, a series of lambda functions packaged in a one-click deployable application that configures everything needed so your S3 objects are submitted automatically to scanii’s content analysis [API](https://docs.scanii.com/v2.1/overview.html). Once the content is processed, you can choose from a couple of different actions: +Scanii Lambda is a deployable AWS Serverless Application Repository application +that submits new Amazon S3 objects to [Scanii](https://scanii.com/) for content +analysis. -1. Tag the content - this is defaulted to on and adds the following tag to objects processed: - 1. `ScaniiId` -> the resource id of the processed content - 2. `ScaniiFindings` -> list of identified findings (content engine dependent) - 3. `ScaniiContentType` -> the identified content type of the file processed -2. Delete the object with findings - this is defaulted to **off** and will delete S3 objects with findings (such as malware or NSFW content) - for a full list of available content identification see https://docs.scanii.com/article/149-how-do-the-different-detection-engines-work +Use it when you want S3 uploads scanned for malware, unsafe language, NSFW +content, or other Scanii findings without writing your own Lambda integration. +The full deployment walkthrough is available in +[Analyze content stored on Amazon S3](https://docs.scanii.com/article/151-how-do-i-analyze-content-stored-on-amazon-s3). -## Working with the source code -The source code for this application is written using Javascript and requires, at least, nodejs 8 to run. Before getting started we strongly advise you to become familiar with the following technologies: +## Deploy on AWS + +[Open Scanii Lambda in the AWS Serverless Application Repository](https://serverlessrepo.aws.amazon.com/#/applications/arn:aws:serverlessrepo:us-east-1:484983087487:applications~UvaSoftware-Scanii-Lambda) +to deploy the published SAM application into your AWS account. + +## What gets deployed + +The application creates: + +1. A submit Lambda function that receives S3 object-created events. +2. A callback Lambda function behind API Gateway that receives Scanii results. +3. IAM permissions for the selected bucket so the functions can read objects, + tag objects, and optionally delete objects with findings. + +The submit function creates a short-lived signed S3 URL, sends that URL to +Scanii's async fetch API, and includes the callback URL. When Scanii finishes +processing the object, it calls the callback function. The callback verifies the +request metadata and runs the configured S3 actions. + +Existing objects are not scanned automatically. Only object-created events sent +to the submit function are processed. + +## Before deploying + +Have these ready: + +1. An existing S3 bucket to monitor. +2. A Scanii API key and secret. +3. The Scanii regional endpoint you want to use. +4. AWS permissions to create Lambda functions, API Gateway resources, IAM roles, + and S3 event notifications. -1. [Amazon S3](https://aws.amazon.com/s3/) -2. [Amazon Lambda](https://aws.amazon.com/lambda/) -3. [AWS Serverless Application Model (SAM) specification](https://github.com/awslabs/serverless-application-model) +Deploy the application in the same AWS Region as the S3 bucket you want to +monitor. -### Building and running tests -Tests utilize Mocha and are triggered into NPM, we provide a makefile to tie everything together: +## Parameters + +| Parameter | Required | Description | +|---|---:|---| +| `bucketName` | Yes | Existing S3 bucket to monitor. | +| `scaniiApiKey` | Yes | Your Scanii API key. | +| `scaniiApiSecret` | Yes | Your Scanii API secret. | +| `scaniiApiEndpoint` | Yes | Regional Scanii API host. The default is `api-us1.scanii.com`. Use the host only, without `https://`. See [Endpoints and regions](https://docs.scanii.com/article/161-endpoints-and-regions). | +| `actionTagObject` | No | Set to `true` to add Scanii result tags to processed objects. Default: `true`. | +| `actionDeleteObjectOnFinding` | No | Set to `true` to delete objects when Scanii reports findings. Default: `false`. | + +For supported endpoint values, see +[Endpoints and regions](https://docs.scanii.com/article/161-endpoints-and-regions). + +## Advanced configuration + +The deployment parameters cover the standard customer-facing settings. The +Lambda functions also read environment variables for callback URL, retry, and +signed URL behavior. See [lib/config.js](lib/config.js) for the full list and +defaults before overriding those values. + +## Required post-deploy step + +After the stack finishes deploying, add an S3 trigger to the submit function. +The function name is: + +```text +-Submit ``` -$ make test + +In the Lambda console: + +1. Open the submit function. +2. Choose **Add trigger**. +3. Select **S3**. +4. Select the bucket from `bucketName`. +5. Set the event type to **All object create events**. +6. Add optional prefix or suffix filters if you only want to scan part of the + bucket. +7. Save the trigger. + +New matching uploads will now be submitted to Scanii. + +## Actions + +### Tag processed objects + +When `actionTagObject` is `true`, the callback function preserves existing tags +and appends these tags: + +| Tag | Value | +|---|---| +| `ScaniiId` | Scanii resource ID for the processed object. | +| `ScaniiFindings` | Space-separated Scanii findings, or `None` when there are no findings. | +| `ScaniiContentType` | Content type reported by Scanii. | + +S3 tag values have length limits, so long finding values are truncated before +they are written. + +### Delete objects with findings + +When `actionDeleteObjectOnFinding` is `true`, the callback function deletes an +object if Scanii returns one or more findings. + +Use this carefully. For a first production deployment, leave deletion disabled, +verify the tags and CloudWatch logs, and then enable deletion after you are +comfortable with the behavior. + +For more detail on Scanii finding categories, see +[How do the different detection engines work?](https://docs.scanii.com/article/149-how-do-the-different-detection-engines-work). + +## Verifying a deployment + +1. Upload a new test object to the monitored bucket. +2. Check the submit function's CloudWatch logs for the Scanii submission ID. +3. Check the callback function's CloudWatch logs for the processing result. +4. If tagging is enabled, verify the S3 object tags. +5. If deletion is enabled and the object has findings, verify the object was + deleted. + +If nothing is submitted, confirm that the S3 trigger exists on +`-Submit`, uses the right bucket, and listens for object-created +events. + +## Working with the source code + +The application is written for the AWS Lambda Node.js 24 runtime. + +Useful commands: + +```bash +npm ci +npm test +make test ``` -### Running the application locally -If you have the SAM CLI (https://github.com/awslabs/aws-sam-cli) installed locally you can run scanii-lambda locally for testing: +If you have the AWS SAM CLI installed, you can run the local API: +```bash +make run ``` -$ make run + +And you can generate a sample S3 event for the submit function: + +```bash +make run-submit-event ``` -## Deploying it -You can deploy this application by clicking [here.](https://serverlessrepo.aws.amazon.com/#/applications/arn:aws:serverlessrepo:us-east-1:484983087487:applications~UvaSoftware-Scanii-Lambda) - please note that after deployment you must manually create a trigger event for the `uvasoftware-scanii-lambda-submit` function for your S3 bucket, under “Add Triggers/S3” and event type `Object Created (All)` +The Scanii API contract is published at +[https://scanii.github.io/openapi/v22/](https://scanii.github.io/openapi/v22/). diff --git a/lib/config.js b/lib/config.js index d1dd1a2..63d1a25 100644 --- a/lib/config.js +++ b/lib/config.js @@ -6,17 +6,38 @@ if (process.env.AWS_SAM_LOCAL !== undefined) { } function defaults() { + // Scanii API credentials. Both handlers require these to authenticate API + // calls and verify callback metadata. CONFIG.KEY = null; CONFIG.SECRET = null; + + // Scanii API host without a scheme. The submit handler adds "https://" when + // constructing the client endpoint. CONFIG.API_ENDPOINT = "api-us1.scanii.com"; + + // Public callback URL passed to Scanii when submitting S3 objects. The SAM + // template sets this to the API Gateway callback route after deployment. CONFIG.CALLBACK_URL = null; + + // Enables writing Scanii result tags back to the processed S3 object. CONFIG.ACTION_TAG_OBJECT = false; + + // Enables deleting S3 objects when Scanii reports one or more findings. CONFIG.ACTION_DELETE_OBJECT = false; + + // Maximum number of Scanii API attempts made by the local retry wrapper. CONFIG.MAX_ATTEMPTS = 10; + + // Maximum randomized delay between retry attempts, in milliseconds. CONFIG.MAX_ATTEMPT_DELAY_MSEC = 30_000; - CONFIG.SIGNED_URL_DURATION = 3600 -// extracting config overwrites from the environment: + // S3 signed URL lifetime, in seconds. This must be long enough for Scanii to + // fetch the object after the submit handler receives the S3 event. + CONFIG.SIGNED_URL_DURATION = 3600; + + // Environment variables override the defaults above. SAM deployment + // parameters set the common customer-facing values, and advanced operators can + // set the remaining variables directly on the Lambda functions. if (process.env.API_KEY) { CONFIG.KEY = process.env.API_KEY; } diff --git a/package-lock.json b/package-lock.json index f33b687..68b0988 100644 --- a/package-lock.json +++ b/package-lock.json @@ -1,12 +1,12 @@ { "name": "scanii-lambda", - "version": "3.0.0", + "version": "3.0.2", "lockfileVersion": 3, "requires": true, "packages": { "": { "name": "scanii-lambda", - "version": "3.0.0", + "version": "3.0.2", "license": "Apache-2.0", "dependencies": { "@aws-sdk/client-s3": "^3.600.0", diff --git a/package.json b/package.json index 33729de..c950e76 100644 --- a/package.json +++ b/package.json @@ -1,7 +1,7 @@ { "name": "scanii-lambda", - "version": "3.0.1", - "description": "Uva Software's Scanii.com Lambda/SAM Integration", + "version": "3.0.2", + "description": "Scanii Lambda/SAM integration for Amazon S3", "main": "index.js", "scripts": { "test": "mocha tests --recursive --exit --no-timeouts" diff --git a/template.yml b/template.yml index 1b2b877..3db3441 100644 --- a/template.yml +++ b/template.yml @@ -26,14 +26,14 @@ Parameters: actionTagObject: Description: Should custom tags be added to S3 objects after processing? Type: String - Default: yes + Default: true AllowedValues: - true - false actionDeleteObjectOnFinding: Description: Should S3 objects be DELETED once a finding is identified? Type: String - Default: no + Default: false AllowedValues: - true - false