Do you want to Train Large scale semantic NLP Models in your Delphi GUI App? This post will get to understand how to use Gensim Python Library using Python4Delphi in Delphi/C++ application and learn the core concepts of Gensim – A Superfast, Proven, Data Streaming, Platform Independent library with some pretrained models for specific domains like legal or health.
Python for Delphi (P4D) is a set of free components that wrap up the Python DLL into Delphi. They let you easily execute Python scripts, create new Python modules and new Python types. You can use Python4Delphi a number of different ways such as:
- Create a Windows GUI around your existing Python app.
- Add Python scripting to your Delphi Windows apps.
- Add parallel processing to your Python apps through Delphi threads.
- Enhance your speed-sensitive Python apps with functions from Delphi for more speed.
Prerequisites.
- If not python and Python4Delphi is not installed on your machine, Check this how to run a simple python script in Delphi application using Python4Delphi sample app
- Open windows open command prompt, and type pip install -U gensim to install GenSim. For more info for Installing Python Modules check here
- First, run the Demo1 project for executing Python script in Python for Delphi. Then load the Texblob sample script in the Memo1 field and press the Execute Script button to see the result. On Clicking Execute Button the script strings are executed using the below code. Go to GitHub to download the Demo1 source.
Gensim Core concepts :
- Document: A document is an object of the text sequence type (commonly known as
strin Python 3). A document could be anything from a short 140 character tweet, a single paragraph (i.e., journal article abstract), a news article, or a book. - Corpus: a collection of documents. Serve 2 purposes.
- Input for training a Model. During training, the models use this training corpus to look for common themes and topics, initializing their internal model parameters.
- Documents to organize. After training, a topic model can be used to extract topics from new documents (documents not seen in the training corpus).Such corpora can be indexed for Similarity Queries, queried by semantic similarity, clustered etc.
- Vector: a mathematically convenient representation of a document.
- Model: an algorithm for transforming vectors from one representation to another.
Gensim Python Library sample script details: The sample scripts helps to understand how the core concepts were implemented for a simple Corpus.
- A Corpus consists of 9 documents where each document consisting of a string.
- Created a set of frequent words where to be ignored by splitting it by white space.
- Get the word count frequencies and just keep the words which is occurring more than once.
- Assign to dictionary in corpora and print the tokens and its id by calling token2id.
- Create the bag-of-word representation for a new document using the doc2bow and convert our entire original corpus to a list of vectors.
- Train using the model ‘tf-idf‘ – transforms vectors from the bag-of-words representation to a vector space
- Transform the “system minors” string from the dictionary using doc2bow
- Transform the whole corpus via TfIdf and index it, in preparation for similarity queries.
Note: Samples used for demonstration were picked from here with only the difference of printing the outputs. You can check the APIs and some more samples from the same place.
You have read the quick overview of Gensim library, download this library from here, and perform NLP tasks quickly with help of models such as word2vec, Latent Dirichlet Allocation Model, FastText Model, etc. Check out Python4Delphi and easily build Python GUIs for Windows using Delphi.