Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,22 @@

All notable changes to `scanii-lambda` are documented here. Versions follow [SemVer](https://semver.org).

## [3.0.2] — Deployment README refresh

### Changed

- Rewrote the README around the AWS Serverless Application Repository customer
deployment flow, including parameters, required post-deploy S3 trigger setup,
result actions, and deployment verification.
- Moved the AWS deployment link into a prominent `Deploy on AWS` section and
moved brittle endpoint enumeration out of the README in favor of the
maintained endpoint documentation.
- Documented the advanced environment configuration defaults inline in
`lib/config.js`.
- Corrected the SAM parameter defaults for `actionTagObject` and
`actionDeleteObjectOnFinding` to use the documented and accepted `true` /
`false` values.

## [3.0.1] — Node.js 24 handler compatibility

### Fixed
Expand Down
166 changes: 144 additions & 22 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,34 +1,156 @@
# Uva Software’s scanii-lambda
A Sam-Packaged AWS Lambda client to the [scanii.com](https://scanii.com) content processing engine. For a detailed walk-through of deploying this application see: https://docs.scanii.com/article/151-how-do-i-analyze-content-stored-on-amazon-s3.
# Scanii Lambda for Amazon S3

## How it works
This is, essentially, a series of lambda functions packaged in a one-click deployable application that configures everything needed so your S3 objects are submitted automatically to scanii’s content analysis [API](https://docs.scanii.com/v2.1/overview.html). Once the content is processed, you can choose from a couple of different actions:
Scanii Lambda is a deployable AWS Serverless Application Repository application
that submits new Amazon S3 objects to [Scanii](https://scanii.com/) for content
analysis.

1. Tag the content - this is defaulted to on and adds the following tag to objects processed:
1. `ScaniiId` -> the resource id of the processed content
2. `ScaniiFindings` -> list of identified findings (content engine dependent)
3. `ScaniiContentType` -> the identified content type of the file processed
2. Delete the object with findings - this is defaulted to **off** and will delete S3 objects with findings (such as malware or NSFW content) - for a full list of available content identification see https://docs.scanii.com/article/149-how-do-the-different-detection-engines-work
Use it when you want S3 uploads scanned for malware, unsafe language, NSFW
content, or other Scanii findings without writing your own Lambda integration.
The full deployment walkthrough is available in
[Analyze content stored on Amazon S3](https://docs.scanii.com/article/151-how-do-i-analyze-content-stored-on-amazon-s3).

## Working with the source code
The source code for this application is written using Javascript and requires, at least, nodejs 8 to run. Before getting started we strongly advise you to become familiar with the following technologies:
## Deploy on AWS

[Open Scanii Lambda in the AWS Serverless Application Repository](https://serverlessrepo.aws.amazon.com/#/applications/arn:aws:serverlessrepo:us-east-1:484983087487:applications~UvaSoftware-Scanii-Lambda)
to deploy the published SAM application into your AWS account.

## What gets deployed

The application creates:

1. A submit Lambda function that receives S3 object-created events.
2. A callback Lambda function behind API Gateway that receives Scanii results.
3. IAM permissions for the selected bucket so the functions can read objects,
tag objects, and optionally delete objects with findings.

The submit function creates a short-lived signed S3 URL, sends that URL to
Scanii's async fetch API, and includes the callback URL. When Scanii finishes
processing the object, it calls the callback function. The callback verifies the
request metadata and runs the configured S3 actions.

Existing objects are not scanned automatically. Only object-created events sent
to the submit function are processed.

## Before deploying

Have these ready:

1. An existing S3 bucket to monitor.
2. A Scanii API key and secret.
3. The Scanii regional endpoint you want to use.
4. AWS permissions to create Lambda functions, API Gateway resources, IAM roles,
and S3 event notifications.

1. [Amazon S3](https://aws.amazon.com/s3/)
2. [Amazon Lambda](https://aws.amazon.com/lambda/)
3. [AWS Serverless Application Model (SAM) specification](https://github.com/awslabs/serverless-application-model)
Deploy the application in the same AWS Region as the S3 bucket you want to
monitor.

### Building and running tests
Tests utilize Mocha and are triggered into NPM, we provide a makefile to tie everything together:
## Parameters

| Parameter | Required | Description |
|---|---:|---|
| `bucketName` | Yes | Existing S3 bucket to monitor. |
| `scaniiApiKey` | Yes | Your Scanii API key. |
| `scaniiApiSecret` | Yes | Your Scanii API secret. |
| `scaniiApiEndpoint` | Yes | Regional Scanii API host. The default is `api-us1.scanii.com`. Use the host only, without `https://`. See [Endpoints and regions](https://docs.scanii.com/article/161-endpoints-and-regions). |
| `actionTagObject` | No | Set to `true` to add Scanii result tags to processed objects. Default: `true`. |
| `actionDeleteObjectOnFinding` | No | Set to `true` to delete objects when Scanii reports findings. Default: `false`. |

For supported endpoint values, see
[Endpoints and regions](https://docs.scanii.com/article/161-endpoints-and-regions).

## Advanced configuration

The deployment parameters cover the standard customer-facing settings. The
Lambda functions also read environment variables for callback URL, retry, and
signed URL behavior. See [lib/config.js](lib/config.js) for the full list and
defaults before overriding those values.

## Required post-deploy step

After the stack finishes deploying, add an S3 trigger to the submit function.
The function name is:

```text
<stack-name>-Submit
```
$ make test

In the Lambda console:

1. Open the submit function.
2. Choose **Add trigger**.
3. Select **S3**.
4. Select the bucket from `bucketName`.
5. Set the event type to **All object create events**.
6. Add optional prefix or suffix filters if you only want to scan part of the
bucket.
7. Save the trigger.

New matching uploads will now be submitted to Scanii.

## Actions

### Tag processed objects

When `actionTagObject` is `true`, the callback function preserves existing tags
and appends these tags:

| Tag | Value |
|---|---|
| `ScaniiId` | Scanii resource ID for the processed object. |
| `ScaniiFindings` | Space-separated Scanii findings, or `None` when there are no findings. |
| `ScaniiContentType` | Content type reported by Scanii. |

S3 tag values have length limits, so long finding values are truncated before
they are written.

### Delete objects with findings

When `actionDeleteObjectOnFinding` is `true`, the callback function deletes an
object if Scanii returns one or more findings.

Use this carefully. For a first production deployment, leave deletion disabled,
verify the tags and CloudWatch logs, and then enable deletion after you are
comfortable with the behavior.

For more detail on Scanii finding categories, see
[How do the different detection engines work?](https://docs.scanii.com/article/149-how-do-the-different-detection-engines-work).

## Verifying a deployment

1. Upload a new test object to the monitored bucket.
2. Check the submit function's CloudWatch logs for the Scanii submission ID.
3. Check the callback function's CloudWatch logs for the processing result.
4. If tagging is enabled, verify the S3 object tags.
5. If deletion is enabled and the object has findings, verify the object was
deleted.

If nothing is submitted, confirm that the S3 trigger exists on
`<stack-name>-Submit`, uses the right bucket, and listens for object-created
events.

## Working with the source code

The application is written for the AWS Lambda Node.js 24 runtime.

Useful commands:

```bash
npm ci
npm test
make test
```

### Running the application locally
If you have the SAM CLI (https://github.com/awslabs/aws-sam-cli) installed locally you can run scanii-lambda locally for testing:
If you have the AWS SAM CLI installed, you can run the local API:

```bash
make run
```
$ make run

And you can generate a sample S3 event for the submit function:

```bash
make run-submit-event
```

## Deploying it
You can deploy this application by clicking [here.](https://serverlessrepo.aws.amazon.com/#/applications/arn:aws:serverlessrepo:us-east-1:484983087487:applications~UvaSoftware-Scanii-Lambda) - please note that after deployment you must manually create a trigger event for the `uvasoftware-scanii-lambda-submit` function for your S3 bucket, under “Add Triggers/S3” and event type `Object Created (All)`
The Scanii API contract is published at
[https://scanii.github.io/openapi/v22/](https://scanii.github.io/openapi/v22/).
25 changes: 23 additions & 2 deletions lib/config.js
Original file line number Diff line number Diff line change
Expand Up @@ -6,17 +6,38 @@ if (process.env.AWS_SAM_LOCAL !== undefined) {
}

function defaults() {
// Scanii API credentials. Both handlers require these to authenticate API
// calls and verify callback metadata.
CONFIG.KEY = null;
CONFIG.SECRET = null;

// Scanii API host without a scheme. The submit handler adds "https://" when
// constructing the client endpoint.
CONFIG.API_ENDPOINT = "api-us1.scanii.com";

// Public callback URL passed to Scanii when submitting S3 objects. The SAM
// template sets this to the API Gateway callback route after deployment.
CONFIG.CALLBACK_URL = null;

// Enables writing Scanii result tags back to the processed S3 object.
CONFIG.ACTION_TAG_OBJECT = false;

// Enables deleting S3 objects when Scanii reports one or more findings.
CONFIG.ACTION_DELETE_OBJECT = false;

// Maximum number of Scanii API attempts made by the local retry wrapper.
CONFIG.MAX_ATTEMPTS = 10;

// Maximum randomized delay between retry attempts, in milliseconds.
CONFIG.MAX_ATTEMPT_DELAY_MSEC = 30_000;
CONFIG.SIGNED_URL_DURATION = 3600

// extracting config overwrites from the environment:
// S3 signed URL lifetime, in seconds. This must be long enough for Scanii to
// fetch the object after the submit handler receives the S3 event.
CONFIG.SIGNED_URL_DURATION = 3600;

// Environment variables override the defaults above. SAM deployment
// parameters set the common customer-facing values, and advanced operators can
// set the remaining variables directly on the Lambda functions.
if (process.env.API_KEY) {
CONFIG.KEY = process.env.API_KEY;
}
Expand Down
4 changes: 2 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions package.json
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{
"name": "scanii-lambda",
"version": "3.0.1",
"description": "Uva Software's Scanii.com Lambda/SAM Integration",
"version": "3.0.2",
"description": "Scanii Lambda/SAM integration for Amazon S3",
"main": "index.js",
"scripts": {
"test": "mocha tests --recursive --exit --no-timeouts"
Expand Down
4 changes: 2 additions & 2 deletions template.yml
Original file line number Diff line number Diff line change
Expand Up @@ -26,14 +26,14 @@ Parameters:
actionTagObject:
Description: Should custom tags be added to S3 objects after processing?
Type: String
Default: yes
Default: true
AllowedValues:
- true
- false
actionDeleteObjectOnFinding:
Description: Should S3 objects be DELETED once a finding is identified?
Type: String
Default: no
Default: false
AllowedValues:
- true
- false
Expand Down
Loading