Skip to content
12 changes: 9 additions & 3 deletions en/api-reference/guides/knowledge-base.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,7 @@ The script below needs Bash, curl, jq, and Python 3. Copy its two code blocks, i
</Step>
</Steps>

To ask a follow-up in the same session, send the previous response's `x-session-id` as `sessionId` in [Create Query Token](/en/api-reference/queries/create-query-token). You can also send an ID of your own: an unknown ID starts a new session under it. Dify ties a session to your API key, not to an end user, so in a multi-user app give each user their own session ID, such as a UUID you generate.
Each query is answered on its own, even when you send an earlier response's `x-session-id` as `sessionId`, so put everything a follow-up needs into its `query`.

## Investigate an Answer {#investigate-an-answer}

Expand Down Expand Up @@ -112,7 +112,9 @@ Unlike a query at `research` depth, which answers within the stream, a research
Send your question as `query` to [Plan Research Task](/en/api-reference/research-tasks/plan-research-task), with the `mode` and `topK` you want to use. It returns the estimated cost, latency, and steps without starting anything.
</Step>
<Step title="Create">
Call [Create Research Task](/en/api-reference/research-tasks/create-research-task) with the same question and settings. To reject the task up front when its estimates go beyond what you'll accept, add `limits`: a task whose estimates exceed them fails with 422. To cap spending while the task runs, add `budgetUsd`. A task stopped by its budget before it starts writing the answer ends with no answer, so set the budget above the plan's `estimates.costUsd.max` when you need one.
Call [Create Research Task](/en/api-reference/research-tasks/create-research-task) with the same question and settings. To reject the task up front when its estimates go beyond what you'll accept, add `limits`: a task whose estimates exceed them fails with 422.

To cap spending while the task runs, add `budgetUsd`. When you need an answer, set it well above the plan's `estimates.costUsd.max`, because each model call has to fit in the budget at the most it could cost. A task that reaches its budget ends as `canceled`.
</Step>
<Step title="Poll">
Call [Get Research Task](/en/api-reference/research-tasks/get-research-task) with the returned task ID until the task ends.
Expand All @@ -131,7 +133,7 @@ A document is the item the **Document** tab shows. It has a document ID, and the
| Document file ID | [List Document Files](/en/api-reference/documents-and-chunks/list-document-files), `documentAssetId` in a citation, or `metadata.documentId` in [List Query Chunks](/en/api-reference/query-records/list-query-chunks), which holds a document file ID despite its name | [Get Document File](/en/api-reference/documents-and-chunks/get-document-file), [Get Document Outline](/en/api-reference/documents-and-chunks/get-document-outline), [Get Document by File](/en/api-reference/documents-and-chunks/get-document-by-file), [Delete Document File](/en/api-reference/documents-and-chunks/delete-document-file), [Delete Document Files in Batch](/en/api-reference/documents-and-chunks/delete-document-files-in-batch) |
| Document ID | [List Documents](/en/api-reference/documents-and-chunks/list-documents), [Get Document by File](/en/api-reference/documents-and-chunks/get-document-by-file), or the address bar of the open document (`/datasets/new/<knowledge base ID>/documents/<document ID>`) | [Get Document](/en/api-reference/documents-and-chunks/get-document), [Update Document Metadata](/en/api-reference/documents-and-chunks/update-document-metadata), [List Document Revisions](/en/api-reference/documents-and-chunks/list-document-revisions), [List Chunks](/en/api-reference/documents-and-chunks/list-chunks), [Get Chunk](/en/api-reference/documents-and-chunks/get-chunk), [List Document Indexing Tasks](/en/api-reference/indexing-tasks/list-document-indexing-tasks), [Get Document Indexing Task](/en/api-reference/indexing-tasks/get-document-indexing-task) |

To show which document a citation came from, pass its `documentAssetId` to [Get Document by File](/en/api-reference/documents-and-chunks/get-document-by-file) with `1` as `document_asset_version`, since every document file is at version `1`.
To show which document a citation came from, pass its `documentAssetId` to [Get Document by File](/en/api-reference/documents-and-chunks/get-document-by-file) with `1` as `document_asset_version`.

To read a document's chunks, pass the document ID and a revision number to [List Chunks](/en/api-reference/documents-and-chunks/list-chunks). For the content in use for retrieval, take `active.revision` from [Get Document](/en/api-reference/documents-and-chunks/get-document). For older content, find the revision in [List Document Revisions](/en/api-reference/documents-and-chunks/list-document-revisions).

Expand Down Expand Up @@ -216,6 +218,8 @@ To sync on a schedule, save one for each source:

The first scheduled sync runs one interval after you save, so sync now as well if you want fresh content right away.

After each import into a scheduled source finishes, read both values again and save the schedule. An import changes the source's `version`, and a schedule still bound to the old one turns itself off at its next due time.

To stop scheduled syncs, save `mode: manual` with `enabled: false`. Setting the source's `status` to `disabled` with [Update Source](/en/api-reference/sources/update-source) only pauses them, and a sync that came due in the meantime runs as soon as the source is active again.

## Re-index and Track Indexing {#track-indexing}
Expand Down Expand Up @@ -254,6 +258,8 @@ Each update can change either the embedding model or the retrieval settings, not

To remove a whole document, delete it in Dify. Over the API, you can delete document files ([Delete Document File](/en/api-reference/documents-and-chunks/delete-document-file), [Delete Document Files in Batch](/en/api-reference/documents-and-chunks/delete-document-files-in-batch)) and sources ([Delete Source](/en/api-reference/sources/delete-source)).

Deleting a document file deletes only the revision built on it, not the document. Avoid deleting the active revision's file. No older revision takes its place, and the document drops out of List Documents and Dify's document list. It isn't deleted, though, and still counts toward your document quota.

Deleting a source also deletes the documents it imported. To keep them, first save the IDs of files with that `source_id` from [List Document Files](/en/api-reference/documents-and-chunks/list-document-files), because their `source_id` becomes `null` once the deletion completes. Then delete the source with `documents=keep`. The kept documents are hidden until the deletion completes, and after that no source updates them.

To confirm a deletion finished, poll [Get Deletion Job](/en/api-reference/deletion-jobs/get-deletion-job) with `job.id` until the job ends.
Expand Down
10 changes: 5 additions & 5 deletions en/api-reference/openapi_service.json
Original file line number Diff line number Diff line change
Expand Up @@ -22476,7 +22476,7 @@
},
"/knowledge-fs/spaces/{knowledge_base_id}/documents/bulk": {
"delete": {
"description": "Requests the deletion of up to 100 document files in one call and reports an outcome for each file.\n\nSend the JSON body with this `DELETE`. If your client omits bodies on DELETE requests, configure it to send the body explicitly.\n\nFiles are sorted by `documentId`, then `expectedRevision`, and each is handled on its own. A file that does not exist, has a stale `expectedRevision`, is already being deleted, or cannot be deleted with this key comes back in `results` as `rejected`, and the other files proceed. An infrastructure failure stops the run and leaves the remaining files `pending`.\n\nTo continue after an infrastructure failure, or after the call fails as a whole, send the same request again with the same `Idempotency-Key`. It returns the original batch with the jobs already accepted and retries only the `pending` files.\n\nThis call creates no bulk task.\n\nFor the deletion flow, see [Delete Documents and Sources](/en/api-reference/guides/knowledge-base#delete-documents-and-sources).",
"description": "Requests the deletion of up to 100 document files in one call and reports an outcome for each file. As with [Delete Document File](/en/api-reference/documents-and-chunks/delete-document-file), avoid files of active revisions.\n\nSend the JSON body with this `DELETE`. If your client omits bodies on DELETE requests, configure it to send the body explicitly.\n\nFiles are sorted by `documentId`, then `expectedRevision`, and each is handled on its own. A file that does not exist, has a stale `expectedRevision`, is already being deleted, or cannot be deleted with this key comes back in `results` as `rejected`, and the other files proceed. An infrastructure failure stops the run and leaves the remaining files `pending`.\n\nTo continue after an infrastructure failure, or after the call fails as a whole, send the same request again with the same `Idempotency-Key`. It returns the original batch with the jobs already accepted and retries only the `pending` files.\n\nThis call creates no bulk task.\n\nFor the deletion flow, see [Delete Documents and Sources](/en/api-reference/guides/knowledge-base#delete-documents-and-sources).",
"operationId": "deleteDocumentFilesInBatch",
"parameters": [
{
Expand Down Expand Up @@ -23471,7 +23471,7 @@
},
"/knowledge-fs/spaces/{knowledge_base_id}/documents/{document_file_id}": {
"delete": {
"description": "Requests the deletion of a document file and returns the deletion job. Every revision built on the file stops returning chunks through [List Chunks](/en/api-reference/documents-and-chunks/list-chunks).\n\nSend the JSON body with this `DELETE`. If your client omits bodies on DELETE requests, configure it to send the body explicitly.\n\nFor the deletion flow, see [Delete Documents and Sources](/en/api-reference/guides/knowledge-base#delete-documents-and-sources).",
"description": "Requests the deletion of a document file and returns the deletion job. Every revision built on the file stops returning chunks through [List Chunks](/en/api-reference/documents-and-chunks/list-chunks).\n\nAvoid deleting the file of a document's active revision. The document is left `failed` with no active revision and drops out of [List Documents](/en/api-reference/documents-and-chunks/list-documents) and Dify's document list. It still counts toward the document quota.\n\nSend the JSON body with this `DELETE`. If your client omits bodies on DELETE requests, configure it to send the body explicitly.\n\nFor the deletion flow, see [Delete Documents and Sources](/en/api-reference/guides/knowledge-base#delete-documents-and-sources).",
"operationId": "deleteDocumentFile",
"parameters": [
{
Expand Down Expand Up @@ -28056,7 +28056,7 @@
"type": "string"
},
"sessionId": {
"description": "ID of a query session, as a UUID. Omit it to start a new session. [Run Query](/en/api-reference/queries/run-query) returns the session ID in its `x-session-id` response header.\n\nA session stores the last 20 queries. It expires after 30 minutes without a query.\n\nAn expired or unknown ID starts a fresh session under that ID, so a UUID you generate yourself stays valid as that session's ID.",
"description": "ID of a query session, as a UUID. Omit it to start a new session. [Run Query](/en/api-reference/queries/run-query) returns the session ID in its `x-session-id` response header.\n\nA session stores the last 20 queries but doesn't carry them into later answers. It expires after 30 minutes without a query.\n\nAn expired or unknown ID starts a fresh session under that ID, so a UUID you generate yourself stays valid as that session's ID.",
"nullable": true,
"type": "string"
}
Expand Down Expand Up @@ -28733,7 +28733,7 @@
"description": "Provide a nonblank `query`.",
"properties": {
"budgetUsd": {
"description": "Spend ceiling in USD. Enforced while the task runs, not at creation. The task is accepted even when the [Plan Research Task](/en/api-reference/research-tasks/plan-research-task) estimate exceeds it. Before each model call, the task checks the call's worst-case cost against the budget, and it is canceled instead of making a call that could pass the budget.\n\nA call can still cost more than its worst-case estimate, so `cost.totalUsd` can end above `budgetUsd`. A budget cancellation before the `generating` stage leaves the task with no answer. Set the ceiling above the plan's `estimates.costUsd.max` when you need an answer at all.",
"description": "Spend ceiling in USD. Enforced while the task runs, not at creation. The task is accepted even when the [Plan Research Task](/en/api-reference/research-tasks/plan-research-task) estimate exceeds it. Before each model call, the task checks the call's worst-case cost against the budget, and it is canceled instead of making a call that could pass the budget.\n\nA call can still cost more than its worst-case estimate, so `cost.totalUsd` can end above `budgetUsd`. A budget cancellation before the `generating` stage leaves the task with no answer. Worst-case costs usually run well above the plan's estimates, so when you need an answer, set the ceiling well above `estimates.costUsd.max`, and higher still for a large `topK`.\n\nA task stopped by its budget ends in stage `canceled`.",
"minimum": 0,
"nullable": true,
"type": "number"
Expand Down Expand Up @@ -38249,7 +38249,7 @@
}
},
"put": {
"description": "Sets a source's sync schedule, replacing the previous one, and returns it.\n\nSaving does not start a sync. The first scheduled sync is due one interval after the update, and each later one an interval after the previous one was queued. To sync now, call [Sync Source](/en/api-reference/source-syncs-and-imports/sync-source).\n\nTo pause scheduled syncs without changing the schedule, set the source's `status` to `disabled` with [Update Source](/en/api-reference/sources/update-source). A sync that came due meanwhile runs when the source is set back to `active`.\n\nFor the read-then-save flow, see [Sync and Schedule Sources](/en/api-reference/guides/knowledge-base#sync-and-schedule-sources).",
"description": "Sets a source's sync schedule, replacing the previous one, and returns it.\n\nSaving does not start a sync. The first scheduled sync is due one interval after the update, and each later one an interval after the previous one was queued. To sync now, call [Sync Source](/en/api-reference/source-syncs-and-imports/sync-source).\n\nAfter each import into the source finishes, save the schedule again with the source's new `version`. An import changes `version`, and a schedule still bound to the old one turns itself off at its next due time.\n\nTo pause scheduled syncs without changing the schedule, set the source's `status` to `disabled` with [Update Source](/en/api-reference/sources/update-source). A sync that came due meanwhile runs when the source is set back to `active`.\n\nFor the read-then-save flow, see [Sync and Schedule Sources](/en/api-reference/guides/knowledge-base#sync-and-schedule-sources).",
"operationId": "updateSourceSyncPolicy",
"parameters": [
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ Go to **Knowledge**, switch to **Agent**, and click **Create**.

Give the knowledge base a name and a description, and set who can view it and connect it to their apps. All of these can be changed later in the knowledge base's settings.

The knowledge base uses your workspace's default models. To make images in your documents searchable, it needs a [vision-capable embedding model](/en/cloud/use-dify/knowledge-base/multimodal-retrieval). Switching to one after you add content re-indexes every document.
The knowledge base uses your workspace's default models. To make images in your documents searchable, use a [vision-capable embedding model](/en/cloud/use-dify/knowledge-base/multimodal-retrieval).

## Add Content

Expand Down
14 changes: 7 additions & 7 deletions en/cloud/use-dify/knowledge-base/test-and-tune.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -23,17 +23,17 @@ Every run stays under **Records**, so you can compare results before and after a

### Golden Questions

A golden question is a question paired with the chunks it should return. Together, your golden questions are the bar every later change has to clear.
Some questions must return the right chunks no matter what you change later, such as the ones your users ask most. Keep those as golden questions, along with the chunks each one should return. Then each time you change a setting or model, run one [evaluation](#run-an-evaluation) to check that they all still pass.

When a run returns the right chunks, click **Keep golden question** under its results, tick the chunks that must come back, and choose whether all of them or any one of them has to return for the question to pass.

To add one without a test run: open the **Quality** page, click **Add golden**, enter the question, describe the chunk you expect, and click **Find evidence** to pick from the closest matches. If you already have a list of questions, import it there from a CSV file (up to 1 MB).

<Info>
A golden question counts only once it has expected chunks. Until you pick them, it's a Draft, and [evaluations](#run-an-evaluation) skip it.
Evaluations skip golden questions with these statuses:

If an expected chunk is deleted, for example when you re-upload its document, the question turns **Stale** until you pick new chunks. Evaluations skip Stale questions too.
</Info>
- **Draft**: it has no expected chunks yet. Edit it and pick them.
- **Stale**: an expected chunk no longer exists, for example because you re-uploaded its document or a sync removed that passage. Edit it and pick new chunks.
- **Paused**: an expected chunk's document is disabled. It counts again once you enable the document.

### Bad Cases

Expand All @@ -49,13 +49,13 @@ When the right chunk is missing or ranked too low, click **Make bad case** under

The rerank model assigns the scores. Try a different one, which may rank the same chunk higher.

Cases wait in **Quality** > **Bad cases**. After a fix, click **Replay** to rerun the question with the current settings. If the result is now right, click **To golden question**, find the chunks it should return, and click **Promote**. The case becomes a golden question and closes. **Ignore** closes a case you won't fix.
Cases wait in **Quality** > **Bad cases**. After a fix, click **Replay** to rerun the question with the current settings. If the result is now right, click **To golden question**, find the chunks it should return, and click **Promote**. **Ignore** closes a case you won't fix.

Bad cases also open on their own when a workflow or Agent searches the knowledge base and misses content it holds, so check the list once your apps are live, not only while you're testing.

## Run an Evaluation {#run-an-evaluation}

After changing a setting or model, go to **Quality** > **Evaluations**, click **Run evaluation**, pick the retrieval mode your apps use, and click **Start evaluation**. It reruns every golden question against the current settings, and passes only when each one finds its expected chunks.
After changing a setting or model, go to **Quality** > **Evaluations**, click **Run evaluation**, pick the retrieval mode your apps use, and click **Start evaluation**. It reruns all active golden questions against the current settings, and passes only when each one finds its expected chunks.

For a question that missed, click its count under **Passages found** to see which expected chunks didn't come back.

Expand Down
Loading
Loading