Context and objectives

The foundation's teams fielded the same questions repeatedly: which calls for projects are open, who qualifies, how to apply, and who to contact. This information lived across web pages and internal documents, spread over four languages (English, French, Dutch, and German). Retrieving and explaining it consistently was extremely time-consuming.

The POC aimed to validate whether:

• An LLM-based assistant could provide fast, accurate access to the right information

• Multilingual support could be handled within a single solution

• The system could run securely inside the foundation's own Azure environment

The broader goal was to lay the groundwork for a scalable solution that could eventually reduce the growing volume of repetitive inquiries.

Approach

1. Scoping and data preparation

The foundation had already defined target personas before the project started. Agilytic used these personas as a key input during workshops to refine the chatbot’s objectives, identify recurring question patterns, and design test cases. Several data sources were identified:

• Website content, extracted as snapshots for each of the four languages

• Internal documents from the foundation's asset management systems

As always, access requirements and infrastructure responsibilities were agreed upon early to avoid delays during development.

2. Building the RAG chatbot on Azure

The solution used Dify as the orchestration layer, running in Docker on an Azure VM, with Azure OpenAI as the model provider. Four language-specific knowledge bases were built from the website snapshots, supplemented by relevant internal documents.

The conversation flow included automatic language detection and branching, so users received answers in their own language without switching manually.

3. Iterative testing and tuning

Internal stakeholders tested the RAG chatbot on Azure over several rounds. Prompts and retrieval parameters were adjusted based on their feedback:

• Reducing overly long answers

• Improving relevance of retrieved passages

• Flagging gaps in the underlying data

Results

The POC confirmed that a multilingual RAG chatbot on Azure can be deployed and used effectively, initially for internal testing.

1. What delivered value

• Response times averaged around 3 seconds, reaching up to 6 seconds for more complex queries

• Users validated the concept and appreciated the multilingual capability

• The solution ran securely within the foundation's own Azure tenant

2. What surfaced as risks

Data completeness directly drove answer quality. Some calls for projects were missing from the website extract, causing the chatbot to return incorrect or outdated information. Some responses were also perceived as too long or insufficiently focused.

Furthermore, without a controlled deployment strategy, costs and abuse risks could escalate.

3. Roadmap to production

1. Data: replace ad-hoc website snapshots with a structured source of truth (data warehouse or database) and define automated update mechanisms

2. Infrastructure: scale compute for higher concurrency, review Azure OpenAI token quotas, and add a managed gateway for SSL, key rotation, rate limiting, and prompt-injection hardening

3. Deployment: phase the rollout from internal use with human oversight to limited external access behind authentication, and finally to public website deployment once stabilized

The foundation views the concept as valuable, particularly for providing information outside business hours. From now on, the primary condition for moving forward is confidence in the underlying data quality.