Context and Objectives

A German IT service provider in the public sector struggled to gain clear insights into their partners' past and ongoing IT projects.

Without proper visibility, they couldn't effectively assess their R&D investments or manage innovation activities.

The stakes were high—poor project landscape visibility meant they could misallocate resources, miss strategic opportunities, and lose their competitive edge.

Approach

We began with data gathering, scraping information from 30 customers' websites using predefined IT keywords. We leveraged SERP API to retrieve documents and pages—a tool that integrates structured search engine results into applications and services.

Next, during document pre-filtering, we identified IT-related publications by analyzing keyword frequency. To enhance data quality, we removed unnecessary elements like tables of contents and references. We employed Hugging Face, an open-source platform offering natural language processing models for text classification, named entity recognition, and question answering.

In the information extraction phase, we classified each document's IT projects using a pre-defined taxonomy.

Finally, we developed a map-based Power BI tool that presents actionable insights for the management team, enabling streamlined analysis and strategic planning.

Throughout the project, we maintained close communication with the client through regular meetings. This ongoing dialogue ensured we met their needs, made timely adjustments, and properly managed potential risks.

Results

The solution provides an efficient pipeline that begins with scraping websites through keywords and advanced Google search. The system then filters and sorts the scraped documents. The proof-of-concept delivers a map view of hundreds of classified documents through PowerBI integration, with potential for scaling to additional partners.

Our tailored solution streamlines document analysis and classification, addressing the client's specific needs. By combining robust processing capabilities with innovative technology, we deliver precise results while optimizing our client's time and resources.