Skip to content

Commit dff8f34

Browse files
JslYoonJslYoon
authored
RHIDP-13056: separating notebooks and lightspeed vector_store (#2861)
* separating notebooks and lightspeed vector_store Signed-off-by: Lucas <lyoon@redhat.com> * notebooks only requires queryDefaults, adding changeset Signed-off-by: Lucas <lyoon@redhat.com> * all files uploaded to lightspeed stack is now .txt Signed-off-by: Lucas <lyoon@redhat.com> * passing tests Signed-off-by: Lucas <lyoon@redhat.com> * clean code Signed-off-by: Lucas <lyoon@redhat.com> * addressing comments Signed-off-by: Lucas <lyoon@redhat.com> * fix comments Signed-off-by: Lucas <lyoon@redhat.com> * adding readme Signed-off-by: Lucas <lyoon@redhat.com> * fixed spelling errors & grammar on readme Signed-off-by: Lucas <lyoon@redhat.com> --------- Signed-off-by: Lucas <lyoon@redhat.com>
1 parent f9dd27d commit dff8f34

16 files changed

Lines changed: 109 additions & 1293 deletions

File tree

‎workspaces/lightspeed/.changeset/chilled-guests-search.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
'@red-hat-developer-hub/backstage-plugin-lightspeed-backend': minor
3+
---
4+
5+
All lightspeed query is now called with rhdh-docs vector_store. Notebooks app-config only now requires queryDefaults model and provider
6+
All files uploaded to lightspeed-stack will be converted to .txt

‎workspaces/lightspeed/app-config.yaml‎

Lines changed: 2 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -21,12 +21,8 @@ lightspeed:
2121
notebooks:
2222
enabled: false
2323
queryDefaults:
24-
model: redhataillama-31-8b-instruct
25-
provider_id: vllm
26-
sessionDefaults:
27-
provider_id: notebooks
28-
embedding_model: ${LLAMA_STACK_EMBEDDING_MODEL}
29-
embedding_dimension: 768
24+
model: ${NOTEBOOKS_QUERY_MODEL}
25+
provider_id: ${NOTEBOOKS_QUERY_PROVIDER_ID}
3026

3127
backend:
3228
# Used for enabling authentication, secret is shared by all backend plugins

‎workspaces/lightspeed/plugins/lightspeed-backend/README.md‎

Lines changed: 3 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -110,15 +110,8 @@ lightspeed:
110110
# Required: Query defaults for RAG queries
111111
# Both model and provider_id must be configured together
112112
queryDefaults:
113-
model: llama3.1-8b-instruct # Model to use for answering queries
114-
provider_id: ollama # AI provider for the query model
115-
116-
# Required: Session defaults for creating vector stores
117-
# All three fields are required when Notebooks is enabled
118-
sessionDefaults:
119-
provider_id: notebooks # Vector store provider ID (must match Llama Stack config)
120-
embedding_model: sentence-transformers/all-mpnet-base-v2 # Model for generating embeddings
121-
embedding_dimension: 768 # Embedding vector dimension (must match model output)
113+
model: ${NOTEBOOKS_QUERY_MODEL} # Model to use for answering queries. Must map to a model available through the provider set in $NOTEBOOKS_QUERY_PROVIDER_ID
114+
provider_id: ${NOTEBOOKS_QUERY_PROVIDER_ID} # AI provider for the query model. Must map to a provider enabled in your Lightspeed config.yaml
122115
123116
# Optional: Chunking strategy for document processing
124117
chunkingStrategy:
@@ -143,11 +136,7 @@ lightspeed:
143136
- **`queryDefaults.model`** _(required)_: The LLM model to use for answering RAG queries. Must be available in the configured provider.
144137
- **`queryDefaults.provider_id`** _(required)_: The AI provider identifier for the query model (e.g., `ollama`, `vllm`). Both `model` and `provider_id` must be configured together.
145138

146-
**Session Defaults** _(required when enabled)_:
147-
148-
- **`sessionDefaults.provider_id`** _(required)_: Vector store provider identifier. Must match a provider configured in your Llama Stack instance (e.g., `notebooks`, `chromadb`). This determines where document embeddings are stored.
149-
- **`sessionDefaults.embedding_model`** _(required)_: The embedding model to use for converting documents to vectors (e.g., `sentence-transformers/all-mpnet-base-v2`). Must be available in Llama Stack.
150-
- **`sessionDefaults.embedding_dimension`** _(required)_: Dimension of the embedding vectors produced by the embedding model. Must match the model's output dimension (commonly `768`, `384`, or `1536`).
139+
> **Important**: The `model` and `provider_id` values must map to a provider and model that are actually enabled in your Lightspeed config.yaml configuration. If the provider or model is not available in Lightspeed, queries will fail. For example, if `openai` enabled in Lightspeed via ENABLE_OPENAI, then model must be available, e.g (model=gpt-4o-mini).
151140

152141
**Chunking Strategy** _(optional)_:
153142

‎workspaces/lightspeed/plugins/lightspeed-backend/__fixtures__/lcsHandlers.ts‎

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -268,6 +268,20 @@ export const lcsHandlers: HttpHandler[] = [
268268
});
269269
}),
270270

271+
// Vector stores list endpoint - returns mock RHDH product docs vector store
272+
http.get(`${LOCAL_LCS_ADDR}/v1/vector-stores`, () => {
273+
return HttpResponse.json({
274+
data: [
275+
{
276+
id: 'vs-rhdh-product-docs',
277+
name: 'rhdh-product-docs',
278+
provider_id: 'notebooks',
279+
metadata: {},
280+
},
281+
],
282+
});
283+
}),
284+
271285
// Catch-all handler for unknown paths
272286
http.all(`${LOCAL_LCS_ADDR}/*`, ({ request }) => {
273287
console.log(`Caught request to unknown path: ${request.url}`);

‎workspaces/lightspeed/plugins/lightspeed-backend/app-config.yaml‎

Lines changed: 0 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -13,13 +13,6 @@
1313
# provider_id: ollama # AI provider for query model (e.g., ollama, vllm)
1414
# model: llama3.1-8b-instruct # Model to use for answering queries
1515
#
16-
# # REQUIRED when enabled: Session defaults for vector stores
17-
# # All three fields are required
18-
# sessionDefaults:
19-
# provider_id: notebooks # Vector store provider ID (must match Llama Stack config)
20-
# embedding_model: sentence-transformers/all-mpnet-base-v2 # Embedding model for documents
21-
# embedding_dimension: 768 # Vector dimension (must match embedding model output)
22-
#
2316
# # OPTIONAL: Chunking strategy for document processing
2417
# chunkingStrategy:
2518
# type: auto # 'auto' (default) or 'static'

‎workspaces/lightspeed/plugins/lightspeed-backend/src/service/constant.ts‎

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,7 @@ export const DEFAULT_CHUNKING_STRATEGY_TYPE = 'auto'; // auto chunking
2222
export const DEFAULT_MAX_CHUNK_SIZE_TOKENS = 512; // 512 tokens
2323
export const DEFAULT_CHUNK_OVERLAP_TOKENS = 50; // 50 tokens
2424
export const DEFAULT_LLAMA_STACK_PORT = 8321; // Llama Stack port
25+
export const DEFAULT_LIGHTSPEED_SERVICE_HOST = '0.0.0.0'; // Lightspeed core service host
2526
export const DEFAULT_LIGHTSPEED_SERVICE_PORT = 8080; // Lightspeed service port
2627
export const DEFAULT_MAX_FILE_SIZE_MB = 20 * 1024 * 1024; // 20MB
2728
export const NOTEBOOKS_SYSTEM_PROMPT =
@@ -36,16 +37,16 @@ Constraints:
3637
Output Format:
3738
1. Summary: A 1-2 sentence high-level answer.
3839
2. Detailed Analysis: A structured breakdown using bullet points.
39-
3. References: A list of sources used.
40+
3. References: A list of sources used. References should be in the format of [Document Title] in a new line for each reference.
4041
4142
Disclaimer: Your answers **MUST** be grounded in the provided documents. If the answer isn't present, state: "I don't know based on the provided documents."
43+
Remember, **ALL** references must be from the provided documents and provided documents only.
4244
Make no mistakes.
4345
`.trim();
4446

4547
/**
4648
* HTTP and networking constants
4749
*/
48-
export const LIGHTSPEED_SERVICE_HOST = '0.0.0.0'; // Lightspeed core service host
4950
export const URL_FETCH_TIMEOUT_MS = 30000; // 30 second timeout for URL fetching
5051
export const USER_AGENT = 'RHDH-AI-Notebooks-Bot/1.0'; // User agent for HTTP requests
5152
export const MAX_URL_CONTENT_SIZE = 10 * 1024 * 1024; // 10MB max for URL fetched content

‎workspaces/lightspeed/plugins/lightspeed-backend/src/service/notebooks/VectorStoresOperator.ts‎

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,16 +82,45 @@ async function handleHttpError(
8282
*
8383
* This class provides the same interface as LlamaStackClient but proxies calls through
8484
* lightspeed-core REST API instead of calling llama stack directly.
85+
*
86+
* Implemented as a singleton to ensure single instance across the application.
8587
*/
8688
export class VectorStoresOperator {
89+
private static instance: VectorStoresOperator | null = null;
8790
private baseURL: string;
8891
private logger: LoggerService;
8992

90-
constructor(lightspeedCoreUrl: string, logger: LoggerService) {
93+
private constructor(lightspeedCoreUrl: string, logger: LoggerService) {
9194
this.baseURL = lightspeedCoreUrl;
9295
this.logger = logger;
9396
}
9497

98+
/**
99+
* Get the singleton instance of VectorStoresOperator
100+
* @param lightspeedCoreUrl - Lightspeed core URL (required on first call)
101+
* @param logger - Logger service (required on first call)
102+
* @returns The singleton instance
103+
*/
104+
static getInstance(
105+
lightspeedCoreUrl: string,
106+
logger: LoggerService,
107+
): VectorStoresOperator {
108+
if (!VectorStoresOperator.instance) {
109+
VectorStoresOperator.instance = new VectorStoresOperator(
110+
lightspeedCoreUrl,
111+
logger,
112+
);
113+
}
114+
return VectorStoresOperator.instance;
115+
}
116+
117+
/**
118+
* Reset the singleton instance (primarily for testing)
119+
*/
120+
static resetInstance(): void {
121+
VectorStoresOperator.instance = null;
122+
}
123+
95124
/**
96125
* Vector Stores API - mirrors LlamaStackClient.vectorStores structure
97126
*/

‎workspaces/lightspeed/plugins/lightspeed-backend/src/service/notebooks/documents/documentService.test.ts‎

Lines changed: 10 additions & 41 deletions
Original file line numberDiff line numberDiff line change
@@ -62,9 +62,10 @@ describe('DocumentService', () => {
6262
},
6363
},
6464
});
65-
operator = new VectorStoresOperator(LIGHTSPEED_CORE_ADDR, logger);
65+
VectorStoresOperator.resetInstance(); // Reset singleton before each test
66+
operator = VectorStoresOperator.getInstance(LIGHTSPEED_CORE_ADDR, logger);
6667
documentService = new DocumentService(operator, logger, config);
67-
sessionService = new SessionService(operator, logger, config);
68+
sessionService = new SessionService(operator, logger);
6869

6970
// Create a test session for document operations
7071
const session = await sessionService.createSession(
@@ -84,7 +85,6 @@ describe('DocumentService', () => {
8485
const fileId = await documentService.uploadFile(
8586
'Test content',
8687
'test-file.txt',
87-
'txt',
8888
);
8989

9090
expect(fileId).toBeDefined();
@@ -93,23 +93,13 @@ describe('DocumentService', () => {
9393

9494
it('should handle upload errors', async () => {
9595
// Mock a failure by passing invalid content
96-
await expect(
97-
documentService.uploadFile('', '', 'txt'),
98-
).resolves.toBeDefined();
96+
await expect(documentService.uploadFile('', '')).resolves.toBeDefined();
9997
});
10098

10199
it('should use correct MIME type based on file type', async () => {
102-
const fileId1 = await documentService.uploadFile(
103-
'{}',
104-
'test.json',
105-
'json',
106-
);
107-
const fileId2 = await documentService.uploadFile(
108-
'text',
109-
'test.txt',
110-
'txt',
111-
);
112-
const fileId3 = await documentService.uploadFile('# MD', 'test.md', 'md');
100+
const fileId1 = await documentService.uploadFile('{}', 'test.json');
101+
const fileId2 = await documentService.uploadFile('text', 'test.txt');
102+
const fileId3 = await documentService.uploadFile('# MD', 'test.md');
113103

114104
expect(fileId1).toBeDefined();
115105
expect(fileId2).toBeDefined();
@@ -119,11 +109,7 @@ describe('DocumentService', () => {
119109

120110
describe('getFileStatus', () => {
121111
it('should get file status for existing document', async () => {
122-
const fileId = await documentService.uploadFile(
123-
'Content',
124-
'Test Doc',
125-
'text',
126-
);
112+
const fileId = await documentService.uploadFile('Content', 'Test Doc');
127113
await documentService.upsertDocument(
128114
sessionId,
129115
'Test Doc',
@@ -149,7 +135,6 @@ describe('DocumentService', () => {
149135
const fileId = await documentService.uploadFile(
150136
'This is test content',
151137
'Test Document',
152-
'text',
153138
);
154139

155140
const result = await documentService.upsertDocument(
@@ -169,7 +154,6 @@ describe('DocumentService', () => {
169154
const fileId1 = await documentService.uploadFile(
170155
'Original content',
171156
'Original Title',
172-
'text',
173157
);
174158
await documentService.upsertDocument(
175159
sessionId,
@@ -181,7 +165,6 @@ describe('DocumentService', () => {
181165
const fileId2 = await documentService.uploadFile(
182166
'Updated content',
183167
'Original Title',
184-
'text',
185168
);
186169
const result = await documentService.upsertDocument(
187170
sessionId,
@@ -199,7 +182,6 @@ describe('DocumentService', () => {
199182
const fileId1 = await documentService.uploadFile(
200183
'Content',
201184
'Original Title',
202-
'text',
203185
);
204186
await documentService.upsertDocument(
205187
sessionId,
@@ -211,7 +193,6 @@ describe('DocumentService', () => {
211193
const fileId2 = await documentService.uploadFile(
212194
'Updated content',
213195
'New Title',
214-
'text',
215196
);
216197
const result = await documentService.upsertDocument(
217198
sessionId,
@@ -230,7 +211,6 @@ describe('DocumentService', () => {
230211
const fileId1 = await documentService.uploadFile(
231212
'Content 1',
232213
'Document 1',
233-
'text',
234214
);
235215
await documentService.upsertDocument(
236216
sessionId,
@@ -242,7 +222,6 @@ describe('DocumentService', () => {
242222
const fileId2 = await documentService.uploadFile(
243223
'Content 2',
244224
'Document 2',
245-
'text',
246225
);
247226
await documentService.upsertDocument(
248227
sessionId,
@@ -265,23 +244,15 @@ describe('DocumentService', () => {
265244
});
266245

267246
it('should filter documents by file type', async () => {
268-
const fileId1 = await documentService.uploadFile(
269-
'Content',
270-
'Text Doc',
271-
'text',
272-
);
247+
const fileId1 = await documentService.uploadFile('Content', 'Text Doc');
273248
await documentService.upsertDocument(
274249
sessionId,
275250
'Text Doc',
276251
'text',
277252
fileId1,
278253
);
279254

280-
const fileId2 = await documentService.uploadFile(
281-
'Content',
282-
'PDF Doc',
283-
'pdf',
284-
);
255+
const fileId2 = await documentService.uploadFile('Content', 'PDF Doc');
285256
await documentService.upsertDocument(
286257
sessionId,
287258
'PDF Doc',
@@ -299,7 +270,6 @@ describe('DocumentService', () => {
299270
const fileId = await documentService.uploadFile(
300271
'Content',
301272
'Test Document',
302-
'text',
303273
);
304274
await documentService.upsertDocument(
305275
sessionId,
@@ -323,7 +293,6 @@ describe('DocumentService', () => {
323293
const fileId = await documentService.uploadFile(
324294
'Content',
325295
'Test Document',
326-
'text',
327296
);
328297
await documentService.upsertDocument(
329298
sessionId,

‎workspaces/lightspeed/plugins/lightspeed-backend/src/service/notebooks/documents/documentService.ts‎

Lines changed: 4 additions & 19 deletions
Original file line numberDiff line numberDiff line change
@@ -22,7 +22,6 @@ import {
2222
DEFAULT_CHUNK_OVERLAP_TOKENS,
2323
DEFAULT_CHUNKING_STRATEGY_TYPE,
2424
DEFAULT_MAX_CHUNK_SIZE_TOKENS,
25-
FILE_TYPE_TO_MIME,
2625
} from '../../constant';
2726
import { SessionDocument, UpsertResult } from '../types/notebooksTypes';
2827
import { VectorStoresOperator } from '../VectorStoresOperator';
@@ -98,27 +97,13 @@ export class DocumentService {
9897
* @returns File ID from the Files API
9998
* @throws Error if upload fails
10099
*/
101-
/**
102-
* Upload a file to the Files API
103-
* @param content - File content as string
104-
* @param title - File title/name
105-
* @param fileType - Optional file type for MIME type detection
106-
* @returns File ID from the Files API
107-
* @throws Error if upload fails
108-
*/
109-
async uploadFile(
110-
content: string,
111-
title: string,
112-
fileType?: string,
113-
): Promise<string> {
100+
async uploadFile(content: string, title: string): Promise<string> {
114101
try {
115102
// Determine MIME type from file type or default to text/plain
116-
const mimeType = fileType
117-
? FILE_TYPE_TO_MIME[fileType] || 'text/plain'
118-
: 'text/plain';
119-
103+
const mimeType = 'text/plain';
104+
const txtFilename = `${title.replace(/\.[^.]+$/, '')}.txt`;
120105
const file = await this.client.files.create({
121-
file: await toFile(Buffer.from(content, 'utf-8'), title, {
106+
file: await toFile(Buffer.from(content, 'utf-8'), txtFilename, {
122107
type: mimeType,
123108
}),
124109
purpose: 'assistants',

‎workspaces/lightspeed/plugins/lightspeed-backend/src/service/notebooks/notebooksRouter.test.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@ import {
2626
resetMockStorage,
2727
} from '../../../__fixtures__/lightspeedCoreHandlers';
2828
import { createNotebooksRouter } from './notebooksRouters';
29+
import { VectorStoresOperator } from './VectorStoresOperator';
2930

3031
const mockUserId = 'user:default/guest';
3132

@@ -53,6 +54,7 @@ describe('Notebooks Router', () => {
5354

5455
beforeEach(async () => {
5556
resetMockStorage();
57+
VectorStoresOperator.resetInstance(); // Reset singleton before each test
5658
const logger = mockServices.logger.mock();
5759
const config = mockServices.rootConfig({
5860
data: {

0 commit comments

Comments
 (0)