CORS Error

[JesJehle]

@mmaelicke I get a cors error while uploading a file. Do you changed the policy?

Access to fetch at 'https://data.deadtrees.earth/api/dev/upload' from origin 'http://localhost:5173' has been blocked by CORS policy: Response to preflight request doesn't pass access control check: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.

[mmaelicke]

No, I did not change anything. I will have a look at it asap

[mmaelicke]

It's not a CORS error, the docker container is not running and the API is not online.

I am looking into the logs to see what is going wrong. It crashed directly after my last update.

[JesJehle]

I tried to upload a TIF file yesterday. Maybe this crashed the process. The file was fine. It said some thing about a missing column of process_path or raw_path.

[mmaelicke]

That was the reason of the update. Not sure yet, what exactly is going wrong. The whole system does not have a really good logging, need to work on that

[mmaelicke]

SHould be solved, at least the Container is healthy and running

[JesJehle]

It looks like the processing crashed again:
No wms yet (uploaded it an hour ago)

[mmaelicke]

The server is running fine, but the file is still pending.
The supabase web hook did not start the processing. I'll have to dig down into the processing to find the problem. This will take time as this part of supabase is largely undocumented, so we can only do that with trial and error

[JesJehle]

All good. One step after the next :)

[mmaelicke]

Checked the logs, and the supabase web hook was never activated. That's weird.

[mmaelicke]

OK, I checked logs again, there was some delay:

just to remind myself to check it. Copied from there supabase json warning

processor-dev-1  | /usr/local/lib/python3.12/site-packages/pydantic/json_schema.py:2099: PydanticJsonSchemaWarning: Default value <bound method BaseModel.schema of <class 'app.SupabaseWebhookPayload'>> is not JSON serializable; excluding default from JSON schema [non-serializable-default]
processor-dev-1  |   warnings.warn(message, PydanticJsonSchemaWarning)
processor-dev-1  | INFO:     172.19.0.1:55990 - "GET /api/dev/openapi.json HTTP/1.0" 200 OK
processor-dev-1  | 2024-03-15 13:22:31,281:INFO - HTTP Request: POST https://ijuphmnaebfdzsfrnsrn.supabase.co/auth/v1/token?grant_type=password "HTTP/1.1 200 OK"
processor-dev-1  | 2024-03-15 13:22:31,346:INFO - HTTP Request: POST https://ijuphmnaebfdzsfrnsrn.supabase.co/auth/v1/logout?scope=global "HTTP/1.1 204 No Content"
processor-dev-1  | 2024-03-15 13:22:31,568:INFO - HTTP Request: GET https://ijuphmnaebfdzsfrnsrn.supabase.co/rest/v1/upload_files_dev?select=uuid&status=eq.pending "HTTP/1.1 200 OK"
processor-dev-1  | INFO:     172.19.0.1:37106 - "GET /api/dev/files/pending HTTP/1.0" 200 OK
processor-dev-1  | 2024-03-15 13:22:49,695:INFO - HTTP Request: GET https://ijuphmnaebfdzsfrnsrn.supabase.co/rest/v1/upload_files_dev?select=uuid&status=eq.pending "HTTP/1.1 200 OK"
processor-dev-1  | 2024-03-15 13:22:49,891:INFO - HTTP Request: GET https://ijuphmnaebfdzsfrnsrn.supabase.co/rest/v1/upload_files_dev?select=%2A&uuid=eq.381f6a52-6cf7-482e-8a4e-ea18ae06ff7e "HTTP/1.1 200 OK"
processor-dev-1  | 2024-03-15 13:22:50,199:INFO - HTTP Request: PATCH https://ijuphmnaebfdzsfrnsrn.supabase.co/rest/v1/upload_files_dev?uuid=eq.381f6a52-6cf7-482e-8a4e-ea18ae06ff7e "HTTP/1.1 200 OK"
processor-dev-1  | [DEBUG]: Final metadata state: user_id='6afa4242-681e-4611-a659-3287d06f6e49' aquisition_date=datetime.datetime(2024, 3, 5, 23, 0) upload_date=datetime.datetime(2024, 3, 15, 10, 55, 59, 886294) file_name='california_cropped_7_2020_7_26.tif' content_type='image/tiff' file_size=627804 raw_path='/data/raw_uploads/381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif' processed_path='/data/processed/381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif' copy_time=0.00554275512695312 uuid='381f6a52-6cf7-482e-8a4e-ea18ae06ff7e' sha256='1f1f259db64021424bcb15bd38e43e5cabffef266c01a9e58fc1a962cd745012' platform=<PlatformEnum.drone: 'drone'> license=<LicenseEnum.cc_by: 'cc-by'> status=<StatusEnum.processed: 'processed'> compress_time=0.3496415615081787 wms_source='https://data.deadtrees.earth/mapserver?LAYERS=381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif' bbox=BoundingBox(left=-118.27508769792131, bottom=36.48377156689809, right=-118.27211543241904, top=36.48617013022948) file_id='381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif'
processor-dev-1  | 2024-03-15 13:22:51,102:DEBUG - Final metadata state: user_id='6afa4242-681e-4611-a659-3287d06f6e49' aquisition_date=datetime.datetime(2024, 3, 5, 23, 0) upload_date=datetime.datetime(2024, 3, 15, 10, 55, 59, 886294) file_name='california_cropped_7_2020_7_26.tif' content_type='image/tiff' file_size=627804 raw_path='/data/raw_uploads/381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif' processed_path='/data/processed/381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif' copy_time=0.00554275512695312 uuid='381f6a52-6cf7-482e-8a4e-ea18ae06ff7e' sha256='1f1f259db64021424bcb15bd38e43e5cabffef266c01a9e58fc1a962cd745012' platform=<PlatformEnum.drone: 'drone'> license=<LicenseEnum.cc_by: 'cc-by'> status=<StatusEnum.processed: 'processed'> compress_time=0.3496415615081787 wms_source='https://data.deadtrees.earth/mapserver?LAYERS=381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif' bbox=BoundingBox(left=-118.27508769792131, bottom=36.48377156689809, right=-118.27211543241904, top=36.48617013022948) file_id='381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif'
processor-dev-1  | [DEBUG]: Updates sent to backend: {'status': 'processed', 'compress_time': 0.3496415615081787, 'processed_path': '/data/processed/381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif', 'wms_source': 'https://data.deadtrees.earth/mapserver?LAYERS=381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif', 'bbox': 'BOX(36.48377156689809 -118.27508769792131, 36.48617013022948 -118.27211543241904)'}
processor-dev-1  | 2024-03-15 13:22:51,102:DEBUG - Updates sent to backend: {'status': 'processed', 'compress_time': 0.3496415615081787, 'processed_path': '/data/processed/381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif', 'wms_source': 'https://data.deadtrees.earth/mapserver?LAYERS=381f6a52-6cf7-482e-8a4e-ea18ae06ff7e_california_cropped_7_2020_7_26.tif', 'bbox': 'BOX(36.48377156689809 -118.27508769792131, 36.48617013022948 -118.27211543241904)'}
processor-dev-1  | 2024-03-15 13:22:51,291:INFO - HTTP Request: PATCH https://ijuphmnaebfdzsfrnsrn.supabase.co/rest/v1/upload_files_dev?uuid=eq.381f6a52-6cf7-482e-8a4e-ea18ae06ff7e "HTTP/1.1 200 OK"
processor-dev-1  | [INFO]: Finished processing 381f6a52-6cf7-482e-8a4e-ea18ae06ff7e in 0.3496415615081787 seconds.
processor-dev-1  | 2024-03-15 13:22:51,488:INFO - HTTP Request: POST https://ijuphmnaebfdzsfrnsrn.supabase.co/rest/v1/logs "HTTP/1.1 201 Created"
processor-dev-1  | 2024-03-15 13:22:51,295:INFO - Finished processing 381f6a52-6cf7-482e-8a4e-ea18ae06ff7e in 0.3496415615081787 seconds.
processor-dev-1  | [INFO]: Dispatching 1 preprocessor threads over /dispatch by API using uuid: 'all'
processor-dev-1  | 2024-03-15 13:22:51,682:INFO - HTTP Request: POST https://ijuphmnaebfdzsfrnsrn.supabase.co/rest/v1/logs "HTTP/1.1 201 Created"
processor-dev-1  | 2024-03-15 13:22:51,490:INFO - Dispatching 1 preprocessor threads over /dispatch by API using uuid: 'all'
processor-dev-1  | INFO:     172.19.0.1:53576 - "POST /api/dev/dispatch?uuid=all HTTP/1.0" 200 OK

[mmaelicke]

until then, I just programmatically invoked https://processor.deadtrees.earth/api/dev/dispatch/<uuid> and that worked fine. The data is processed.

Link to docs: https://processor.deadtrees.earth/api/dev/docs#/default/dispatch_dispatch__uuid__post

[JesJehle]

@mmaelicke I am calling the webhook within a supabase subscription, but get the same cors error. Is the server down again?

  const callWebhook = async (payload: any) => {
    const webhookURL =
      "https://processor.deadtrees.earth/api/dev/dispatch/" + payload.new.uuid;
    const webhookResponse = fetch(webhookURL, {
      method: "POST",
      headers: {
        "Content-Type": "application/json",
      },
      body: JSON.stringify({
        type: "INSERT",
        schema: "public",
        table: "upload_files_dev",
        record: payload.new,
      }),
    });
  };

  useEffect(() => {
    const channel = supabase
      .channel("upload_files_dev")
      .on(
        "postgres_changes",
        {
          event: "*",
          schema: "public",
          table: "upload_files_dev",
        },
        (payload) => {
          console.log("Change received!", payload);
          if (
            payload.eventType === "INSERT" &&
            payload.new.status === "pending"
          ) {
            console.log("calling webhook");
            const webhookResponse = callWebhook(payload);
            console.log("webhook res", webhookResponse);
          }
          fetchData();
        },
      )
      .subscribe();
    fetchData();
    return () => {
      supabase.removeChannel(channel);
    };
  }, [supabase]);

Error:

Access to fetch at 'https://processor.deadtrees.earth/api/dev/dispatch/2c18a61f-a328-4baa-8203-7f4b4c93d06c' from origin 'http://localhost:5173' has been blocked by CORS policy: No 'Access-Control-Allow-Origin' header is present on the requested resource. If an opaque response serves your needs, set the request's mode to 'no-cors' to fetch the resource with CORS disabled.
DataProvider.tsx:36 
 
POST https://processor.deadtrees.earth/api/dev/dispatch/2c18a61f-a328-4baa-8203-7f4b4c93d06c net::ERR_FAILED 500 (Internal Server Error)