Haystack describes the flow of data through components with named inputs and outputs. Search and generation can be run together or debugged in parts.
How to find a problem link
The answer of the assistant with reference to the manual looks convincing until the operator opens the source. Sometimes the cause of the error is in the search, and sometimes the model has distorted the correctly found paragraph. In Haystack, these steps can be described by separate components and check the intermediate results. Save for the test the input question, the documents found and the text passed to the generation. Then, after replacing the model, you will see which part of the process has really changed.
Two branches for the training base
Provide an internal reference where requests for settings go to the documentation index, and questions about supported formats are checked against a small table. Start with the usual question for each branch. Then make a mixed request. See if the selected path matches the rules and what happens if documents are not found. For a negative example, ask about a function that is not in any source.
In this test, it is useful to display the names of the components and the short reason for the route. It is not necessary to keep the content of each real appeal for this.
What to measure when replacing a component
Keep the same set of inputs. Replace the search or way to prepare the context and compare the output before the model is called. Measure the time of each step so that a slow external request is not mistaken for a delay in the entire algorithm. If you are using a reprocessing cycle, set an exit condition and check that the wrong input does not start infinite work.
Suitable tasks
- Check search output before generating a response
- Collect branching for different types of requests
Strengths
- Conveyor allows branches and cycles
- Components have separate inputs and outputs
Limitations
- Configure connections and data formats in code
- When a pipeline uses external services, test failures at each step
Selection and conditions
- Free access
- Yes
- Trial period
- Not confirmed
- No sign-up required
- Not confirmed
- Russian interface
- Not confirmed
- Local installation
- Yes
- API
- Not confirmed
“Not confirmed” means there is not enough information. The feature may still be available; check the developer’s terms.
- Input formats
- Document, Text
- Result
- Data, Text
Terms
The library is available for self-hosting. The cost of cloud hosting, models, and infrastructure depends on the components you use. Configure connections and data formats in code.From task to first result
Practical example
- Task
- Check search output before generating a response
- What to prepare
- Two versions of the instruction manual and questions with known sources.
Task brief or prompt
Prepare an index with version and section fields. Find supporting fragments, return the source document and position.
Expected result
Issuance with the correct version of the instruction and available source.
How to check the result
- Check the search without generating a response.
- Make sure that the index update eliminates the outdated edition.
An editorial teaching example of a task brief, not the result of hands-on testing of the service. Check that the required mode is available, that you have rights to the source material and that the intended use of the result is permitted.
Check details
Official sources
This description is based on the developer’s material. Features, prices and availability may change; check the primary source before use.
