Context and objectives

A German IT service provider in the public sector struggled to gain clear insights into their partners' past and ongoing IT projects. Without proper visibility, they couldn't effectively assess their R&D investments or manage innovation activities.

The stakes were high: a lack of project intelligence and poor landscape visibility meant they could

• misallocate resources,

• miss strategic opportunities,

• and lose their competitive edge.

Approach

1. Data collection

We began with data gathering, scraping information from 30 customers' websites using predefined IT keywords. We leveraged SERP API, a tool that integrates structured search engine results into applications and services, to retrieve documents and pages.

2. Data quality

Next, during document pre-filtering, we identified IT-related publications by analyzing keyword frequency. To enhance data quality, we removed unnecessary elements like tables of contents and references. We employed Hugging Face, an open-source platform offering natural language processing models for text classification, named entity recognition, and question answering.

In the information extraction phase, we classified each document's IT projects using a pre-defined taxonomy.

3. Actionable data

Finally, we developed a map-based Power BI tool that presents actionable insights for the management team, enabling streamlined analysis and strategic planning with enhanced project intelligence.

Throughout the project, we maintained close communication with the client through regular meetings. This ongoing dialogue ensured we met their needs, made timely adjustments, and properly managed potential risks.

Results

The solution provides an efficient pipeline that:

• scrapes multiple websites using keywords and advanced Google search,

• filters and sorts the scraped documents,

• displays hundreds of classified documents in a map view through Power BI integration, with the potential to scale to additional partners.

Our solution streamlines document analysis and classification, addressing the client's specific needs for better project intelligence. By combining robust processing capabilities with innovative technology, we deliver precise results while optimizing our client's time and resources.