loading page

Code Generation from Flowchart using Optical Character Recognition & Large Language Model
  • Aryaman Darda,
  • Reetu Jain
Aryaman Darda

Corresponding Author:[email protected]

Author Profile
Reetu Jain
Author Profile


Students find it difficult to learn the syntax of programming languages when they are new to coding. Some students get discouraged and even quit their coding journey because of having trouble with memorizing all the syntactic rules of a programming language. Flowcharts are an effective way to help students visualize the logic of an algorithm developed to solve a problem. But some students find it challenging to convert their flowcharts into executable code and even when they do, some get frustrated because of the errors they encounter while running the code they have written. So students who are not well versed with the syntax of a programming language can face difficulty debugging their programs even when they understand the logic behind the algorithms. This research aimed at developing a tool that can assist young students with less coding experience to learn programming. An intuitive user interface for the application was developed using a python library called gradio. The functionality of taking flowchart images as user inputs and providing equivalent python code with a brief explanation was incorporated on the UI. Text from the uploaded flowchart image is extracted in 3 steps which include feature extraction using deep learning models like ResNet and VGG, sequence labelling using Long Short-Term Memory networks and decoding the sequence labels using Connectionist Temporal Classification algorithms. The extracted text is then provided to an open source large language model with 70 billion parameters called Llama-2-Chat for the generation of the python code. The generated python code is displayed on the gradio UI along with a brief explanation of the code. The user can copy the code and run it on the IDE of their choice. The application was evaluated with a test dataset of several flowchart images. The generated python codes for the test flowchart images were executed in a code editor and the produced outputs were found to be correct. The generated code got executed without any errors for 75 % of the flowchart images from the test dataset.
18 Apr 2024Submitted to TechRxiv
24 Apr 2024Published in TechRxiv