Skip to content

Commit 7db4bed

Browse files
JslYoonyangcao77
JslYoon
and
yangcao77
authored
[RHIDP-11602] [RHIDP-11600] [RHIDP-11605] AI Notebooks Developer Preview Backend (#2499)
* feat(lightspeed): AI Notebooks with persistent conversations Signed-off-by: Lucas <lyoon@redhat.com> * update readme Signed-off-by: Lucas <lyoon@redhat.com> * addressing lots of comments Signed-off-by: Lucas <lyoon@redhat.com> * refactor & cleanup Signed-off-by: Lucas <lyoon@redhat.com> * cleanup & yarn.lock Signed-off-by: Lucas <lyoon@redhat.com> * more test scripts Signed-off-by: Lucas <lyoon@redhat.com> * cleanup code upsert document & addressing comments Signed-off-by: Lucas <lyoon@redhat.com> * final upsert changes Signed-off-by: Lucas <lyoon@redhat.com> * moving constant to constant.ts Signed-off-by: Lucas <lyoon@redhat.com> * Update workspaces/lightspeed/plugins/lightspeed-backend/src/service/notebooks/notebooksRouters.ts Co-authored-by: Stephanie Cao <yangcao@redhat.com> * Update workspaces/lightspeed/plugins/lightspeed-backend/src/service/notebooks/documents/documentHelpers.ts Co-authored-by: Stephanie Cao <yangcao@redhat.com> * Update workspaces/lightspeed/plugins/lightspeed-backend/src/service/notebooks/documents/documentHelpers.ts Co-authored-by: Stephanie Cao <yangcao@redhat.com> * final cleanup Signed-off-by: Lucas <lyoon@redhat.com> * final changes Signed-off-by: Lucas <lyoon@redhat.com> --------- Signed-off-by: Lucas <lyoon@redhat.com> Co-authored-by: Stephanie Cao <yangcao@redhat.com>
1 parent 4b8c03b commit 7db4bed

30 files changed

Lines changed: 6840 additions & 23 deletions

‎workspaces/lightspeed/.changeset/poor-ads-roll.md‎

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
'@red-hat-developer-hub/backstage-plugin-lightspeed-backend': major
3+
'@red-hat-developer-hub/backstage-plugin-lightspeed-common': minor
4+
---
5+
6+
Implement AI Notebooks session, document, and query service

‎workspaces/lightspeed/app-config.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,11 @@ app:
55
organization:
66
name: Red Hat
77

8+
# Disable AI Notebooks feature by default
9+
lightspeed:
10+
aiNotebooks:
11+
enabled: false
12+
813
backend:
914
# Used for enabling authentication, secret is shared by all backend plugins
1015
# See https://backstage.io/docs/auth/service-to-service-auth for

‎workspaces/lightspeed/plugins/lightspeed-backend/README.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,3 +62,92 @@ permission:
6262
policies-csv-file: /some/path/rbac-policy.csv
6363
policyFileReload: true
6464
```
65+
66+
### AI Notebooks (Developer Preview)
67+
68+
AI Notebooks is an experimental feature that enables document-based conversations with Retrieval-Augmented Generation (RAG).
69+
70+
For user-facing feature documentation, see the [Lightspeed Frontend README](../lightspeed/README.md#ai-notebooks-developer-preview).
71+
72+
#### Prerequisites
73+
74+
AI Notebooks requires a **Llama Stack service** to be running. Llama Stack provides the vector database, embeddings, and RAG capabilities.
75+
76+
For Llama Stack setup and configuration, refer to the [Llama Stack documentation](https://github.com/llamastack/llama-stack).
77+
78+
#### Configuration
79+
80+
To enable AI Notebooks, add the following configuration to your `app-config.yaml`:
81+
82+
```yaml
83+
lightspeed:
84+
aiNotebooks:
85+
enabled: true # Enable AI Notebooks feature (default: false)
86+
87+
# Required when enabled: Llama Stack service configuration
88+
llamaStack:
89+
port: 8321 # Llama Stack API endpoint (required, commonly 8321)
90+
91+
# Optional embedding configuration
92+
embeddingModel: sentence-transformers/nomic-ai/nomic-embed-text-v1.5 # (default shown)
93+
embeddingDimension: 768 # Embedding vector dimension (default: 768)
94+
vectorIo:
95+
providerId: rhdh-docs # Vector store provider ID (default: rhdh-docs)
96+
97+
# Optional: File processing timeout (default: 30000ms = 30 seconds)
98+
fileProcessingTimeoutMs: 30000
99+
100+
# Optional: Chunking strategy for document processing
101+
chunkingStrategy:
102+
type: auto # 'auto' or 'static' (default: auto)
103+
# For 'static' chunking:
104+
maxChunkSizeTokens: 512 # Maximum tokens per chunk (default: 512)
105+
chunkOverlapTokens: 50 # Overlap between chunks (default: 50)
106+
```
107+
108+
**Configuration Options**:
109+
110+
- **`enabled`**: Enable or disable the AI Notebooks feature (default: `false`)
111+
- **`llamaStack.port`**: Port of the Llama Stack service (default: `8321`)
112+
- **`llamaStack.embeddingModel`**: Model used for generating embeddings (default: `sentence-transformers/nomic-ai/nomic-embed-text-v1.5`)
113+
- **`llamaStack.embeddingDimension`**: Dimension of embedding vectors (default: `768`)
114+
- **`llamaStack.vectorIo.providerId`**: Vector store provider in Llama Stack config (default: `rhdh-docs`)
115+
- **`fileProcessingTimeoutMs`**: Timeout for file processing in milliseconds (default: `30000`)
116+
- **`chunkingStrategy.type`**: Document chunking strategy - `auto` (automatic) or `static` (fixed size) (default: `auto`)
117+
- **`chunkingStrategy.maxChunkSizeTokens`**: Maximum chunk size in tokens for static chunking (default: `512`)
118+
- **`chunkingStrategy.chunkOverlapTokens`**: Token overlap between chunks for static chunking (default: `50`)
119+
120+
#### API Endpoints
121+
122+
When enabled, AI Notebooks exposes the following REST API endpoints:
123+
124+
- **Health Check**:
125+
- `GET /lightspeed/ai-notebooks/health` - Health check endpoint
126+
127+
- **Sessions**:
128+
- `POST /lightspeed/ai-notebooks/v1/sessions` - Create a new session
129+
- `GET /lightspeed/ai-notebooks/v1/sessions` - List all sessions
130+
- `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Get session details
131+
- `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Update session
132+
- `DELETE /lightspeed/ai-notebooks/v1/sessions/:sessionId` - Delete session
133+
134+
- **Documents**:
135+
- `POST /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/upload` - Upload document
136+
- `GET /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents` - List documents
137+
- `PUT /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId` - Update document
138+
- `DELETE /lightspeed/ai-notebooks/v1/sessions/:sessionId/documents/:documentId` - Delete document
139+
140+
- **Queries**:
141+
- `POST /lightspeed/ai-notebooks/v1/sessions/:sessionId/query` - Query documents with RAG
142+
143+
#### Permission Framework Support for AI Notebooks
144+
145+
When RBAC is enabled, users need the following permission to use AI Notebooks:
146+
147+
```CSV
148+
p, role:default/team_a, lightspeed.notebooks.use, update, allow
149+
150+
g, user:default/<your-user-name>, role:default/team_a
151+
```
152+
153+
Add this to your `rbac-policy.csv` file along with the existing lightspeed permissions.

‎workspaces/lightspeed/plugins/lightspeed-backend/__fixtures__/llamaStackHandlers.ts‎

Lines changed: 226 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,226 @@
1+
/*
2+
* Copyright Red Hat, Inc.
3+
*
4+
* Licensed under the Apache License, Version 2.0 (the "License");
5+
* you may not use this file except in compliance with the License.
6+
* You may obtain a copy of the License at
7+
*
8+
* http://www.apache.org/licenses/LICENSE-2.0
9+
*
10+
* Unless required by applicable law or agreed to in writing, software
11+
* distributed under the License is distributed on an "AS IS" BASIS,
12+
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
13+
* See the License for the specific language governing permissions and
14+
* limitations under the License.
15+
*/
16+
17+
import { http, HttpResponse, type HttpHandler } from 'msw';
18+
19+
import { DEFAULT_LLAMA_STACK_PORT } from '../src/service/constant';
20+
21+
export const LLAMA_STACK_ADDR = `http://0.0.0.0:${DEFAULT_LLAMA_STACK_PORT}`;
22+
23+
// Mock session data
24+
export const mockSession1 = {
25+
id: 'session-1',
26+
name: 'Test Session 1',
27+
embedding_model: 'sentence-transformers/nomic-ai/nomic-embed-text-v1.5',
28+
embedding_dimension: 768,
29+
provider_id: 'rhdh-docs',
30+
metadata: {
31+
user_id: 'user:default/guest',
32+
name: 'Test Session 1',
33+
description: 'Test description',
34+
created_at: '2024-01-01T00:00:00.000Z',
35+
updated_at: '2024-01-01T00:00:00.000Z',
36+
category: 'test',
37+
project: 'test-project',
38+
document_ids: [],
39+
conversation_id: null,
40+
},
41+
};
42+
43+
export const mockSession2 = {
44+
id: 'session-2',
45+
name: 'Test Session 2',
46+
embedding_model: 'sentence-transformers/nomic-ai/nomic-embed-text-v1.5',
47+
embedding_dimension: 768,
48+
provider_id: 'rhdh-docs',
49+
metadata: {
50+
user_id: 'user:default/guest',
51+
name: 'Test Session 2',
52+
description: 'Another test',
53+
created_at: '2024-01-02T00:00:00.000Z',
54+
updated_at: '2024-01-02T00:00:00.000Z',
55+
document_ids: ['doc-1'],
56+
conversation_id: 'conv-1',
57+
},
58+
};
59+
60+
export const mockFile1 = {
61+
id: 'file-1',
62+
created_at: 1704067200,
63+
status: 'completed' as const,
64+
attributes: {
65+
document_id: 'test-document',
66+
user_id: 'user:default/guest',
67+
title: 'Test Document',
68+
session_id: 'session-1',
69+
source_type: 'text',
70+
created_at: '2024-01-01T00:00:00.000Z',
71+
},
72+
};
73+
74+
// In-memory storage for tests
75+
const vectorStores = new Map<string, any>();
76+
const files = new Map<string, any>();
77+
const vectorStoreFiles = new Map<string, any[]>();
78+
79+
export function resetMockStorage() {
80+
vectorStores.clear();
81+
files.clear();
82+
vectorStoreFiles.clear();
83+
}
84+
85+
export const llamaStackHandlers: HttpHandler[] = [
86+
// Create vector store
87+
http.post(`${LLAMA_STACK_ADDR}/v1/vector_stores`, async ({ request }) => {
88+
const body = (await request.json()) as any;
89+
const id = `vs-${Date.now()}`;
90+
const vectorStore = {
91+
id,
92+
name: body.name,
93+
embedding_model: body.embedding_model,
94+
embedding_dimension: body.embedding_dimension,
95+
provider_id: body.provider_id,
96+
metadata: body.metadata || {},
97+
};
98+
vectorStores.set(id, vectorStore);
99+
vectorStoreFiles.set(id, []);
100+
return HttpResponse.json(vectorStore);
101+
}),
102+
103+
// Get vector store
104+
http.get(`${LLAMA_STACK_ADDR}/v1/vector_stores/:id`, ({ params }) => {
105+
const { id } = params;
106+
const vectorStore = vectorStores.get(id as string);
107+
if (!vectorStore) {
108+
return HttpResponse.json(
109+
{ error: 'Vector store not found' },
110+
{ status: 404 },
111+
);
112+
}
113+
return HttpResponse.json(vectorStore);
114+
}),
115+
116+
// Update vector store
117+
http.post(
118+
`${LLAMA_STACK_ADDR}/v1/vector_stores/:id`,
119+
async ({ params, request }) => {
120+
const { id } = params;
121+
const body = (await request.json()) as any;
122+
const vectorStore = vectorStores.get(id as string);
123+
if (!vectorStore) {
124+
return HttpResponse.json(
125+
{ error: 'Vector store not found' },
126+
{ status: 404 },
127+
);
128+
}
129+
const updated = { ...vectorStore, ...body };
130+
vectorStores.set(id as string, updated);
131+
return HttpResponse.json(updated);
132+
},
133+
),
134+
135+
// Delete vector store
136+
http.delete(`${LLAMA_STACK_ADDR}/v1/vector_stores/:id`, ({ params }) => {
137+
const { id } = params;
138+
vectorStores.delete(id as string);
139+
vectorStoreFiles.delete(id as string);
140+
return HttpResponse.json({ success: true });
141+
}),
142+
143+
// List vector stores
144+
http.get(`${LLAMA_STACK_ADDR}/v1/vector_stores`, () => {
145+
return HttpResponse.json({
146+
data: Array.from(vectorStores.values()),
147+
});
148+
}),
149+
150+
// Create file
151+
http.post(`${LLAMA_STACK_ADDR}/v1/files`, async () => {
152+
const id = `file-${Date.now()}`;
153+
const file = {
154+
id,
155+
created_at: Math.floor(Date.now() / 1000),
156+
purpose: 'assistants',
157+
};
158+
files.set(id, file);
159+
return HttpResponse.json(file);
160+
}),
161+
162+
// Delete file
163+
http.delete(`${LLAMA_STACK_ADDR}/v1/files/:id`, ({ params }) => {
164+
const { id } = params;
165+
files.delete(id as string);
166+
return HttpResponse.json({ success: true });
167+
}),
168+
169+
// Create vector store file
170+
http.post(
171+
`${LLAMA_STACK_ADDR}/v1/vector_stores/:vectorStoreId/files`,
172+
async ({ params, request }) => {
173+
const { vectorStoreId } = params;
174+
const body = (await request.json()) as any;
175+
176+
const vectorStoreFile = {
177+
id: body.file_id,
178+
created_at: Math.floor(Date.now() / 1000),
179+
status: 'completed' as 'in_progress' | 'completed',
180+
attributes: body.attributes || {},
181+
};
182+
183+
const storeFiles = vectorStoreFiles.get(vectorStoreId as string) || [];
184+
storeFiles.push(vectorStoreFile);
185+
vectorStoreFiles.set(vectorStoreId as string, storeFiles);
186+
187+
return HttpResponse.json(vectorStoreFile);
188+
},
189+
),
190+
191+
// List vector store files
192+
http.get(
193+
`${LLAMA_STACK_ADDR}/v1/vector_stores/:vectorStoreId/files`,
194+
({ params }) => {
195+
const { vectorStoreId } = params;
196+
const storeFiles = vectorStoreFiles.get(vectorStoreId as string) || [];
197+
return HttpResponse.json({ data: storeFiles });
198+
},
199+
),
200+
201+
// Get vector store file
202+
http.get(
203+
`${LLAMA_STACK_ADDR}/v1/vector_stores/:vectorStoreId/files/:fileId`,
204+
({ params }) => {
205+
const { vectorStoreId, fileId } = params;
206+
const storeFiles = vectorStoreFiles.get(vectorStoreId as string) || [];
207+
const file = storeFiles.find(f => f.id === fileId);
208+
if (!file) {
209+
return HttpResponse.json({ error: 'File not found' }, { status: 404 });
210+
}
211+
return HttpResponse.json(file);
212+
},
213+
),
214+
215+
// Delete vector store file
216+
http.delete(
217+
`${LLAMA_STACK_ADDR}/v1/vector_stores/:vectorStoreId/files/:fileId`,
218+
({ params }) => {
219+
const { vectorStoreId, fileId } = params;
220+
const storeFiles = vectorStoreFiles.get(vectorStoreId as string) || [];
221+
const filtered = storeFiles.filter(f => f.id !== fileId);
222+
vectorStoreFiles.set(vectorStoreId as string, filtered);
223+
return HttpResponse.json({ success: true });
224+
},
225+
),
226+
];

‎workspaces/lightspeed/plugins/lightspeed-backend/app-config.yaml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,3 +2,7 @@
22
#lightspeed:
33
# servicePort: 8080 # OPTIONAL: Port for lightspeed service (default: 8080)
44
# systemPrompt: <custom_system_prompt> # OPTIONAL: Override default RHDH system prompt
5+
#
6+
# # AI Notebooks (Developer Preview) - Disabled by default
7+
# aiNotebooks:
8+
# enabled: false # Set to true to enable AI Notebooks feature

0 commit comments

Comments
 (0)