Internal server error on production site

[ianhinder]

web_1  | KeyError: 'controlledName'
web_1  | [2025-09-30 08:35:33,261] INFO in _internal: 130.88.96.147 - - [30/Sep/2025 08:35:33] "GET / HTTP/1.0" 500 -
web_1  | Traceback (most recent call last):
web_1  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2213, in __call__
web_1  |     return self.wsgi_app(environ, start_response)
web_1  |   File "/usr/local/lib/python3.8/dist-packages/werkzeug/middleware/proxy_fix.py", line 183, in __call__
web_1  |     return self.app(environ, start_response)
web_1  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2193, in wsgi_app
web_1  |     response = self.handle_exception(e)
web_1  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 2190, in wsgi_app
web_1  |     response = self.full_dispatch_request()
web_1  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1486, in full_dispatch_request
web_1  |     rv = self.handle_user_exception(e)
web_1  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1484, in full_dispatch_request
web_1  |     rv = self.dispatch_request()
web_1  |   File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1469, in dispatch_request
web_1  |     return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
web_1  |   File "/app/application.py", line 253, in index
web_1  |     people = get_people()
web_1  |   File "/app/application.py", line 73, in get_people
web_1  |     "interests": [skill["controlledName"] for skill in persondata["skills"]]
web_1  |   File "/app/application.py", line 73, in <listcomp>
web_1  |     "interests": [skill["controlledName"] for skill in persondata["skills"]]

My suspicion is that a CapX update has changed the API. @aharwood2 ?

[ianhinder]

Looks like this. But I am only calling /api/skills/getAllGrouped, which shouldn't have been affected by this. @DrAtbas Was the schema of the data returned by getAllGrouped changed?

The format of data I am expecting is in the mock data here.

[DrAtbas]

Looks like this. But I am only calling /api/skills/getAllGrouped, which shouldn't have been affected by this. @DrAtbas Was the schema of the data returned by getAllGrouped changed?

The format of data I am expecting is in the mock data here.

The test server works ok.

We have updated the endpoint to be marked with [FromQuery]

Here is the response I get from the test server.

[
  {
    "name": "Mavis Ledger",
    "skills": [
      {
        "skillTagId": 12,
        "name": "Cybersecurity"
      },
      {
        "skillTagId": 9,
        "name": "Kubernetes"
      },
      {
        "skillTagId": 2,
        "name": "JavaScript"
      },
      {
        "skillTagId": 5,
        "name": "AWS"
      },
      {
        "skillTagId": 14,
        "name": "Business Analysis"
      }
    ]
  },
  {
    "name": "Nigel Overfetch-Nelson",
    "skills": [
      {
        "skillTagId": 4,
        "name": "Azure"
      },
      {
        "skillTagId": 12,
        "name": "Cybersecurity"
      },
      {
        "skillTagId": 14,
        "name": "Business Analysis"
      },
      {
        "skillTagId": 7,
        "name": "DevOps"
      },
      {
        "skillTagId": 11,
        "name": "Data Analysis"
      }
    ]
  },
  {
    "name": "Clive Bugworthy",
    "skills": [
      {
        "skillTagId": 4,
        "name": "Azure"
      },
      {
        "skillTagId": 5,
        "name": "AWS"
      },
      {
        "skillTagId": 7,
        "name": "DevOps"
      },
      {
        "skillTagId": 11,
        "name": "Data Analysis"
      },
      {
        "skillTagId": 14,
        "name": "Business Analysis"
      }
    ]
  },
  {
    "name": "Janet Nullington",
    "skills": [
      {
        "skillTagId": 9,
        "name": "Kubernetes"
      },
      {
        "skillTagId": 4,
        "name": "Azure"
      },
      {
        "skillTagId": 5,
        "name": "AWS"
      },
      {
        "skillTagId": 6,
        "name": "Agile"
      },
      {
        "skillTagId": 9,
        "name": "Kubernetes"
      }
    ]
  },
  {
    "name": "Tina Breakaway",
    "skills": [
      {
        "skillTagId": 1,
        "name": "C#"
      },
      {
        "skillTagId": 8,
        "name": "Docker"
      },
      {
        "skillTagId": 5,
        "name": "AWS"
      },
      {
        "skillTagId": 10,
        "name": "Machine Learning"
      },
      {
        "skillTagId": 4,
        "name": "Azure"
      }
    ]
  },
  {
    "name": "Bingo McTrousers",
    "skills": [
      {
        "skillTagId": 3,
        "name": "SQL"
      },
      {
        "skillTagId": 4,
        "name": "Azure"
      },
      {
        "skillTagId": 2,
        "name": "JavaScript"
      },
      {
        "skillTagId": 11,
        "name": "Data Analysis"
      },
      {
        "skillTagId": 6,
        "name": "Agile"
      }
    ]
  },
  {
    "name": "Ankle Goblin",
    "skills": [
      {
        "skillTagId": 11,
        "name": "Data Analysis"
      },
      {
        "skillTagId": 11,
        "name": "Data Analysis"
      },
      {
        "skillTagId": 6,
        "name": "Agile"
      },
      {
        "skillTagId": 7,
        "name": "DevOps"
      },
      {
        "skillTagId": 8,
        "name": "Docker"
      }
    ]
  },
  {
    "name": "Gravy Commander",
    "skills": [
      {
        "skillTagId": 9,
        "name": "Kubernetes"
      },
      {
        "skillTagId": 7,
        "name": "DevOps"
      },
      {
        "skillTagId": 7,
        "name": "DevOps"
      },
      {
        "skillTagId": 3,
        "name": "SQL"
      },
      {
        "skillTagId": 3,
        "name": "SQL"
      }
    ]
  },
  {
    "name": "Lemon Lasso",
    "skills": [
      {
        "skillTagId": 7,
        "name": "DevOps"
      },
      {
        "skillTagId": 14,
        "name": "Business Analysis"
      },
      {
        "skillTagId": 1,
        "name": "C#"
      },
      {
        "skillTagId": 8,
        "name": "Docker"
      },
      {
        "skillTagId": 6,
        "name": "Agile"
      }
    ]
  },
  {
    "name": "Soggy Apple Nibbler",
    "skills": [
      {
        "skillTagId": 11,
        "name": "Data Analysis"
      },
      {
        "skillTagId": 8,
        "name": "Docker"
      },
      {
        "skillTagId": 10,
        "name": "Machine Learning"
      },
      {
        "skillTagId": 8,
        "name": "Docker"
      },
      {
        "skillTagId": 7,
        "name": "DevOps"
      }
    ]
  },
  {
    "name": "Eggplant Acrobat",
    "skills": [
      {
        "skillTagId": 3,
        "name": "SQL"
      },
      {
        "skillTagId": 13,
        "name": "Project Management"
      },
      {
        "skillTagId": 14,
        "name": "Business Analysis"
      },
      {
        "skillTagId": 12,
        "name": "Cybersecurity"
      },
      {
        "skillTagId": 5,
        "name": "AWS"
      }
    ]
  },
  {
    "name": "Cheddar Swoosh",
    "skills": [
      {
        "skillTagId": 6,
        "name": "Agile"
      },
      {
        "skillTagId": 2,
        "name": "JavaScript"
      },
      {
        "skillTagId": 6,
        "name": "Agile"
      },
      {
        "skillTagId": 3,
        "name": "SQL"
      },
      {
        "skillTagId": 5,
        "name": "AWS"
      }
    ]
  },
  {
    "name": "Lumpy Sprinkles",
    "skills": [
      {
        "skillTagId": 8,
        "name": "Docker"
      },
      {
        "skillTagId": 8,
        "name": "Docker"
      },
      {
        "skillTagId": 9,
        "name": "Kubernetes"
      },
      {
        "skillTagId": 6,
        "name": "Agile"
      },
      {
        "skillTagId": 11,
        "name": "Data Analysis"
      }
    ]
  }
]

The schema should be the same; we opted for the Data Transfer (edit: auto-corrected to Transformation) Object route to avoid any issues with the existing programs.

[ianhinder]

Is that getAllGrouped? What happened to the controlledName key in the skill dict? The last release was https://github.com/UoMResearchIT/CapX/blob/6ebe006071ceb4d7deb6e2eb9261f19cc07c2ca4/PPMTool.API/Endpoints/Skills.cs, in August, and the code for the skills endpoint is quite different. In your "data transfer / transformation" thing, have you only selected the name and id attributes, whereas before it was returning everything?

[ianhinder]

Probably we should have a test instance of RSESkillsGraph which points to the test CapX, so these things are caught before CapX is released.

[DrAtbas]

Is that getAllGrouped? What happened to the controlledName key in the skill dict? The last release was https://github.com/UoMResearchIT/CapX/blob/6ebe006071ceb4d7deb6e2eb9261f19cc07c2ca4/PPMTool.API/Endpoints/Skills.cs, in August, and the code for the skills endpoint is quite different. In your "data transfer / transformation" thing, have you only selected the name and id attributes, whereas before it was returning everything?

Yes, that is GetAllGrouped.

Yes, we plan to stick with the Data Transfer Object (DTOs) in the latest release to ensure that if anything in the Database changes, no one else's code breaks. This is because we previously dumped all the data we had, and if the database changes, the responses would also change.

If you'd like, we can expand on the DTOs.

[ianhinder]

I see - yes, that makes sense. Should I just change the skills graph code to use name rather than controlledName?

[DrAtbas]

I see - yes, that makes sense. Should I just change the skills graph code to use name rather than controlledName?

Yes, you can do that; the DTOs should never be changed. We can add additional information, but the naming will be constant. Will that fix the issue? This is because, in the dummy response you provided, there is significantly more information that we are omitting. Do you also need them?