Skip to content

add initial version of dataframe analysis engine capable of filtering…#34

Open
Rajaramnr19 wants to merge 1 commit intomainfrom
users/rajaram/dataframe-analyzer
Open

add initial version of dataframe analysis engine capable of filtering…#34
Rajaramnr19 wants to merge 1 commit intomainfrom
users/rajaram/dataframe-analyzer

Conversation

@Rajaramnr19
Copy link
Copy Markdown

Initial level of implementation for the dataframe analysis engine

Changes Made

  1. Added virtual environment to run the python code
  2. Added requirements.txt to setup the virtual environment in the other PCs.
  3. Added a gRPC server in python to get dataframes and user question
  4. Implemented the server code to generate the python code to filter the dataframe based on the user question with the help of OpenAI LLM
  5. Implemented logic to generate python code that will return the answer to the user question in the form of text and dataframe
  6. Added a gRPC client to test the server logic
  7. Tested the server logic with the spartans data 2023

How to Test?

  1. Added Spartans - 2023 Group Challenge Tracker.xlsx file which can be used to test the logic
  2. Run server.py
  3. Run client.py. Question should be added to the client.py itself in line 55
  4. server.py should process the request from the client and will return the data
  5. Data will be returned in the json format

Known issues/Limitations/Pending Features

  1. Only filtering the dataframe and returning the result as a text or a dataframe is handled. Visualizing the data as graph is not yet handled
  2. LLM prompt message needs some correction. The decision made by the LLM on the format of the filtered information (text or dataframe) needs improvement
  3. When same question is asked multiple times, the answer format can vary. For example, for a specific question, one time it can return the answer as text. If you rerun, it might return the same answer as a dataframe. LLM prompt message needs improvement
  4. LabVIEW side client APIs are yet to be handled. Will proceed with the LabVIEW side client handling after getting an initial review to ensure that I am doing the right thing
  5. Running the gRPC in the Azure or Docker container is yet to be implemented. Will proceed with these activities after getting initial review

… information from a dataframe and returning to client
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

dataframe analysis engine: AI based dataframe analyzer

2 participants