Introduction

Structured data extraction from text has plenty of uses in data analysis and myriad applications, including building predictive machine learning models. Named entity recognition (NER) is a clear example of this: it consists of transforming unstructured, messy text into structured data as a result of identifying, extracting, and categorizing relevant information.

NER has been traditionally conducted using traditional natural language processing (NLP) libraries, but these models usually require huge training datasets and arduous fine-tuning to make them learn to recognize custom entities. Thankfully, Scikit-LLM arose to bring large language model (LLM) reasoning capabilities to the table and help build standardized machine learning pipelines in a few simple steps, all on a variety of target tasks beyond just text classification, including custom NER.

Running Named Entity Recognition with Scikit-LLM

This example can be run entirely in a Google Colab notebook.…