MidcurveNN

Midcurve by Neural Networks

MidcurveNN is a project aimed at solving the challenging problem of finding the midcurve of a 2D closed shape using neural networks. The primary goal is to transform a closed polygon, represented by a set of points or connected lines, into another set of points or connected lines, allowing for the possibility of open or branched polygons in the output. To run the project, follow the provided instructions in the "Instructions to Run" section.

• Goal: Given a 2D closed shape (closed polygon), find its midcurve (polyline, closed or open).
• Input: Set of points or set of connected lines, non-intersecting, simple, convex, closed polygon.
• Output: Another set of points or set of connected lines, open/branched polygons possible.

If you are interested in working/contributing to this project voluntarily, do have a look at the issues

Repository Structure

src/
├── config.py                         # Global configuration
├── data/raw/                         # Raw .dat/.mid coordinate files (shared by all)
├── utils/                            # Shared data prep, plotting, metrics
│
├── image_based/                      # Phase I: raster/bitmap approaches
│   ├── data/                         # Shared generated image data (one copy, no duplication)
│   │   ├── image-pairs/              # PNG pairs, used by simple/cnn/dense/denoiser
│   │   ├── unet-splits/train|test/   # UNet split PNGs
│   │   └── images-combo/train|val|test/  # Side-by-side JPGs for pix2pix/img2img
│   ├── simpleencoderdecoder/         # Dense baseline
│   ├── cnnencoderdecoder/            # CNN encoder-decoder
│   ├── denseencoderdecoder/          # Fully connected
│   ├── denoiserencoderdecoder/       # Denoising autoencoder
│   ├── unet/                         # UNet 2-stage (best image model)
│   ├── pix2pix/                      # Pix2Pix GAN (Keras)
│   ├── img2img/                      # Pix2Pix (PyTorch)
│   └── kaggle/                       # Kaggle notebooks
│
├── geometry_based/                   # Phase III: graph/geometric approaches
│   ├── graph_transformer/            # Non-auto-regressive Graph Transformer ← primary
│   ├── finetuned_graph_transformer/  # Pretrained Graphormer fine-tuned on midcurves
│   └── gnnencoderdecoder/            # Legacy GNN stub (reference)
│
├── text_based/                       # Phase II: LLM/seq2seq (implemented)
│   ├── data/brep/                    # 4 base BRep JSON shapes (I, L, T, Plus)
│   ├── data/csvs/                    # CSV train/test/val splits (993 rows)
│   ├── utils/                        # BRep data pipeline (generate CSVs, visualize)
│   ├── finetuning/                   # QLoRA fine-tuning pipeline (Qwen/Gemma/Mistral)
│   │   └── results/                  # evaluation_results.csv (output of evaluate.py)
│   ├── codeT5/                       # CodeT5 notebooks
│   │   └── results/                  # evaluation CSV (populated after training)
│   ├── ludwig/                       # Ludwig framework notebooks
│   ├── prompt/                       # Few-shot prompting + LLM comparison
│   └── nemotron3/                    # Nemotron-Mini-4B: HF SFTTrainer / Unsloth / few-shot
│       ├── hf_sft_trainer.py         # HFSFTTrainer: QLoRA via HuggingFace SFTTrainer
│       ├── unsloth_trainer.py        # UnslothTrainer: Unsloth-accelerated QLoRA
│       ├── fewshot_prompter.py       # FewShotPrompter: 2-shot inference, no fine-tuning
│       ├── train.py / train_unsloth.py  # Entry-point wrappers
│       └── run_demo.py               # Quick end-to-end demo
│
├── image_based/testing/              # Phase I tests
│   └── test_image_based.py
├── geometry_based/testing/           # Phase III tests
│   └── test_geometry_based.py
├── text_based/testing/               # Phase II tests
│   └── test_text_based.py
└── benchmark.py                      # Cross-approach benchmark (image / geometry / text)

Each of image_based/, geometry_based/, and text_based/ has an analysis_report.md documenting known bugs, design risks, and accuracy recommendations for that approach (no code changes applied yet).

Two other top-level folders sit alongside src/:

• publications/ holds conference/journal submission archives, presentation sources, and supporting material for the papers and talks listed under Publications/Talks below, including prior related research (e.g. the author's Midsurface PhD work).
• references/ holds locally-used prompt templates and research notes for this project.

Instructions to Run

cd src
conda env create -f environment.yml
conda activate midcurvenn

# Generate training data for all approaches
# Outputs: image_based/data/image-pairs/, unet-splits/, images-combo/, text_based/data/sequences.json
python utils/prepare_data.py

# --- Image-based (Phase I) ---
python image_based/simpleencoderdecoder/main_simple_encoderdecoder.py

# Best image model (UNet)
cd image_based/unet && python train.py && python test.py
# Each approach saves a results/results_grid.png (3 rows × 7 cols: Input | GT | Predicted)

# --- Geometry-based (Phase III) ---
cd geometry_based/graph_transformer
python main_graph_transformer.py

# Fine-tune pretrained Graphormer (Phase III-b)
cd geometry_based/finetuned_graph_transformer
pip install transformers>=4.35
python train.py

# --- Text-based / LLM (Phase II) ---
cd text_based/utils
python create_brep_csvs.py          # regenerate CSV dataset from base shapes

cd ../finetuning
python data_validator.py            # validate CSV data
python train.py                     # QLoRA fine-tune Qwen/Gemma/Mistral
python run_pipeline.py --full       # full pipeline in one command
python model_server.py --port 8000  # serve via FastAPI

# --- Nemotron-Mini-4B (Phase II, nemotron3) ---
cd text_based/nemotron3
python hf_sft_trainer.py            # Approach 1: HuggingFace SFTTrainer QLoRA
python unsloth_trainer.py           # Approach 2: Unsloth-accelerated QLoRA (PEFT fallback)
python fewshot_prompter.py          # Approach 3: few-shot base-model inference (no fine-tuning)

# --- Tests & Benchmark ---
cd src
python -m pytest                                              # all 3 approach suites (via pytest.ini)
python -m pytest image_based/testing/test_image_based.py -v  # Phase I only
python -m pytest geometry_based/testing/test_geometry_based.py -v  # Phase III only
python -m pytest text_based/testing/test_text_based.py -v    # Phase II only
python benchmark.py                                   # cross-approach benchmark

Thoughts

What is it?

Graph Summarization/Dimension-Reduction/Compression: Reducing a large graph to a smaller graph preserving its underlying structure, similar to text summarization, which attempts to keep the essence.

Representation Issue

• Shapes can not be modeled as sequences. Although polygon shape L may appear as sequence of points, it is not.
• All shapes can not be drawn without lifting a pencil, which is possible for sequences. Say, Shapes like Y or concentric O, cannot be modeled as sequences. So, Midcurve transformation cannot be modeled as Sequence 2 Sequence network.
• How to represent a geometric figure to feed to any Machine/Deep Learning problem, as they need numeric data in vector form?
• How to convert geometric shapes (even after restricting the domain to 2D linear profile shapes) to vectors?
• Closest data structure is graph, but thats predominantly topological and not geometrical. Meaning, graphs represent only connectivity and not spatial positions. Here, nodes would have coordinates and arcs would have curved-linear shape. So, even Graph Neural Networks, which convolute neighbors around a node and pool the output to generate vectors, are not suitable.
• Need RnD to come up with a way to generate geometric-graph embedding that will convolute at nodes (having coordinates) around arcs (having geometry, say, a set of point coordinates), then pool them to form a meaningful representation. Real crux would be how to formulate pooling to aggregate all incident curves info plus node coordinates info into a single number!!

Variable Lengths Issue

• This is a dimension-reduction problem wherein, in 2D, the input constitutes the sketch profile (parametrically 2D), while the output represents the midcurve (parametrically 1D). Input points are ordered, mostly forming a closed loop known as a manifold. Conversely, output points may lack a defined order and can exhibit branches, referred to as non-manifold structures.
• It poses a variable input and variable output challenge, given that the number of points and lines differs in the input and output.
• This presents a network 2 network problem (not Sequence to Sequence) with variable-sized inputs and outputs.
• For Encoder-Decoder networks like those in Tensorflow, fixed-length inputs are necessary. Introducing variable lengths poses a significant hurdle as padding with a distinct unused value, such as 0,0, is not feasible, considering it could represent a valid point.
• The limitation with Seq2Seq models is notable; the input polygons and output branched midcurves are not linearly connected. They may exhibit loops or branches, necessitating further consideration for a more suitable solution. (More details on limitations below)
• Instead of adopting a point list as input/output, let's consider the well-worked format of images. These images are standardized to a constant size, say 64x64 pixels. The sketch profile is represented as a color profile in a bitmap (black and white only), and similarly, the midcurve appears in the output bitmap. This image-based format allows for the application of LSTM encoder-decoder Seq2Seq models. To diversify training data, one can introduce variations by shifting, rotating, and scaling both input and output. The current focus is on 2D sketch profiles, limited to linear segments within a single, simple polygon without holes.
• The vectorization process involves representing each point as 2 floats/ints. Consequently, the total input vector for a polygon with 'm' points is 2m floats/ints. For closed polygons, the first point is repeated as the last one. The output is a vector of 2n points, repeating the last point in the case of a closed figure. Preparing training data involves using data files from MIDAS. For scalability, input and output can be scaled with different factors, and entries can be randomly shuffled. Determining the maximum number of points in a profile establishes a fixed length for both input and output.
• Resources and further exploration for Seq2Seq models can be found at Tensorflow Seq2Seq Tutorial, Seq2Seq Video Tutorial, A Neural Representation of Sketch Drawings, and Sketch RNN GitHub Repository.
• Additionally, consider incorporating plotting capabilities to visualize polygons and their midcurves, facilitating easy debugging and testing of unseen figures.

Dilution to Images

• Images of geometric shapes address both, representation as well as variable-size issue. Big dilution is that, true geometric shapes are like Vector images, whereas images used here would be of Raster type. Approximation has crept in.

• Even after modeling, the predicted output needs to be post-processed to bring to geometric form. Challenging again.

• Thus, this project is divided into three phases:

  • Phase I: Image to Image transformation learning
    • Img2Img: i/o fixed size 100x100 bitmaps
    • Populate many by scaling/rotating/translating both io shapes within the fixed size
    • Use Encoder Decoder like Semantic Segmentation or Pix2Pix of IMages to learn dimension reduction
  • Phase III: Geometry to Geometry transformation learning
    • Build both, input and output polyline graphs with (x,y) coordinates as node features and edges with node id pairs mentioned. For poly-lines, edges being lines, no need to store geometric intermediate points as features, else for curves, store say, sampled fixed 'n' points.
    • Build Image-Segmentation like Encoder-Decoder network, given Graph Convolution Layers from DGL in place of usual Image-based 2D convolution layer, in the usual pytorch encoder-decoder model.
    • Generate variety of input-output polyline pairs, by using geometric transformations (and not image transformations as done in Phase I).
    • See if Variational Graph Auto-Encoders https://github.com/dmlc/dgl/tree/master/examples/pytorch/vgae can help.
  • Phase II: Using Large Language Models (LLMs)
    • With representation of Profile and Midcurve in form of text, json-brep, so that LLMs can be leveraged
    • Prepapre instruct-based fine-tuning dataset which can be used using Ludwig or classical Hugging Face way of fine-tuning

• Phase I is fully implemented with 7 encoder-decoder variants. Four are the primary focus: Simple AE, Dense AE, CNN AE (CoordConv, stride-2 skip connections), and Denoising AE (2-stage training). UNet (2-stage, CoordConv, weighted BCE) is the strongest but an extended variant.

• Phase II is implemented and currently the best performer (text_based/): QLoRA fine-tuning of Qwen2.5-7B achieves MAE=0.78, PSR=98%, Hausdorff=2.1 on the 4-shape benchmark. Nemotron-Mini-4B in a 2-shot setting achieves PSR=85.7%, topology accuracy=0.83, fitting in ~2 GB VRAM. BRep JSON with explicit Lines/Segments topology solves the branching serialization problem.

• Phase III is implemented as a non-auto-regressive Graph Transformer trained from scratch (geometry_based/graph_transformer/), and a second variant fine-tunes a pretrained Graphormer HuggingFace model (geometry_based/finetuned_graph_transformer/). Comprehensive evaluation is ongoing (preliminary status).

Paper: "Talk like a graph: encoding graphs for large language models" surveys graph-to-text representations. We leverage a geometry representation similar to 3D B-rep (Boundary representation), in 2D:

{
	'ShapeName': 'I',
	'Profile': [(5.0, 5.0), (10.0, 5.0), (10.0, 20.0), (5.0, 20.0)],
	'Midcurve': [(7.5, 5.0), (7.5, 20.0)],
	'Profile_brep': {
				'Points': [(5.0, 5.0), (10.0, 5.0), (10.0, 20.0),(5.0, 20.0)], # list of (x,y) coordinates
				'Lines': [[0, 1], [1, 2], [2, 3], [3, 0]], # list of point ids (ie index in the Points list)
                'Segments': [[0, 1, 2, 3]] # list of line ids (ie index in Lines list)
				},
	'Midcurve_brep': {
				'Points': [(7.5, 5.0), (7.5, 20.0)],
				'Lines': [[0, 1]],
                'Segments': [[0]]
				},				
}

Column information is as below:

• ShapeName (text): name of the shape. Just for visualization/reports.
• Profile(text): List of Points coordinates (x,y) representing outer profile shape, typically closed.
• Midcurve(text): List of Points coordinates (x,y) representing inner midcurve shape, typically open.
• Profile_brep(text): Dictionary in Brep format representing outer profile shape, typically closed.
• Midcurve_brep(text): Dictionary in Brep format representing inner midcurve shape, typically open.

Each Segment is a continuous list of lines. In case of, say Midcurve-T, as there is a intersection, we can treat each line in a separate segment. In case of 'Profile O', there will be two segments, one for outer lines and another for inner lines. Each line is list of points, for now, linear. Points is list of coordinates (x,y), later can be (x,y,z).

Once we have this brep representations of both, profile and the corresponding midcurve, in the text form, then we can try various machine translation approaches or LLM based fine tuning or few-shots prompt engineering.

One major advantage of text based method over image based method is that image output still has stray pixels, cleaning which will be a complex task. But text method has exact points. It may just give odd lines, which can be removed easily.

Publications/Talks

• Vixra paper MidcurveNN: Encoder-Decoder Neural Network for Computing Midcurve of a Thin Polygon, viXra.org e-Print archive, viXra:1904.0429 http://vixra.org/abs/1904.0429
• ODSC proposal https://confengine.com/odsc-india-2019/proposal/10090/midcurvenn-encoder-decoder-neural-network-for-computing-midcurve-of-a-thin-polygon
• CAD Conference 2021, Barcelona, pages 223-225 http://www.cad-conference.net/files/CAD21/CAD21_223-225.pdf
• CAD & Applications 2022 Journal paper 19(6) http://www.cad-journal.net/files/vol_19/CAD_19(6)_2022_1154-1161.pdf
• Google Developers Dev Library https://devlibrary.withgoogle.com/products/ml/repos/yogeshhk-MidcurveNN
• Medium story Geometry, Graphs and GPT talks about using LLMs (Large Language Models) to see if geometry serialized as line-list can predict the midcurve.
• Kaggle LLM-dataset and its Ludwig fine-tuning Notebook. The Image-dataset and its Simple Encode Decoder
• ICONIEA 2024 (IIT Kharagpur) extended abstract: "Midcurve Computation using Large Language Models", presented at International Conference on Intelligent and Innovative Endeavors in Applied Research, 2024
• ICCCIT 2025: "Computing Midcurve with Multi-Layer and Convolutional Neural Networks", published in IEEE Xplore (publications/MidcurveNN_ICCCIT2025.zip)
• Medium blog Nemotron-Mini-4B for Midcurve: describes the 3-approach Nemotron pipeline (HF SFTTrainer, Unsloth, few-shot), results (PSR=85.7%, topology=0.83), and deployment on consumer hardware
• Journal paper (in preparation): "MidcurveNN: A Tri-Paradigm Neural Framework for Geometric Dimension Reduction", IEEEtran format covering all 3 phases (publications/Midcurve_LaTeX/Main_Paper_MidcurveNN_Comprehensive.tex)

Citations

• Boussuge, Flavien, & Marc, Raphaël. (2021, October 9). Mid-Curve Completion Using Convolutional Neural Network. 29th International Meshing Roundtable (IMR), Virtual Conference.

Regarding the state of the art, the closest work related to this paper is   from   Kulkarni[28].   
Kulkarni   proposed MidcurveNN, an encoder-decoder neural network to extract mid-curves from 
polygonal 2D shapes. The principle is to train the network with both a pixel image of the 
polygonal shape and of the final desired mid-curves. Although in an early  stage  of  research,  
the  network  is  able  to  produce reasonably  well  the mid-curves  of  simple  L-shaped polygons.  
The  limitations  of  this  work  remain  in  the noisiness of the produced results. 
It has not been tested on a large diversity of shapes and is performed on the full shape 
globally potentially requiring a high-resolution pixel grid for large models.

[28]Y.H. Kulkarni, MIDCURVENN: Encoder-decoder neural network for computing midcurve of a thin polygon, 
in: Open Data Sci. Conf.,2019

Disclaimer:

Author (yogeshkulkarni@yahoo.com) gives no guarantee of the results of the program. It is just a fun script. Lot of improvements are still to be made. So, don’t depend on it at all.