Skip to content

Commit d707a0d

Browse files
sanchitmonga22sanchitmonga22
committed
Address PR #461 review comments (31 fixes across 18 files)
Triage and fixes for all 47 unique review comments from PRs #460/#461 (MetalRT backend + iOS demo polish). Full per-comment status in comments/PR_461_comments.md. Summary of fixes applied: Build / packaging - Package.swift: gate RABackendMetalRTBinary (zero-checksum) behind metalrtRemoteBinaryAvailable flag; no broken remote target in default SPM resolution. - build-ios.sh: SC2155-safe local TMPSTRIP with fail-fast on mktemp. MetalRT C++ backend - rac_llm_metalrt.cpp: stream cancellation now honored (client_cancelled flag), terminal callback guarded, RAC_ERROR_STREAM_CANCELLED returned on cancel. - rac_vlm_metalrt.cpp: default-init struct fields, reject null/empty model_path, validate RGB dims before int cast + w*h*4 allocation (both non-streaming and streaming paths), strdup null-check + OOM. - rac_stt_metalrt.cpp + rac_tts_metalrt.cpp: default-init loaded=false, strdup null-check + OOM. - rac_backend_metalrt_register.cpp: VLM unregister now uses RAC_CAPABILITY_VISION_LANGUAGE (was TEXT_GENERATION). Core commons - lifecycle_manager.cpp: set state to IDLE after auto-unload. - vlm_component.cpp: stat/S_ISDIR guard before convention-based load. Swift SDK - CppBridge.swift: shutdown() synchronously waits for all 6 actor destroys via Task.detached + DispatchSemaphore, preventing use-after-free on shutdown race. - RunAnywhere+TextGeneration.swift: thinkingTokens/responseTokens apportioned by char ratio (preserves total); inputTokens falls back to promptLength/4 when C++ doesn't report prompt tokens. - RunAnywhere+VoiceSession.swift: turnCompleted gated on turnSucceeded flag; error path no longer emits success. - MetalRTRuntime/MetalRT.swift: capabilities set now includes .vlm. - MetalRTRuntime/include/rac_types.h: RAC_STRING_VIEW now evaluates arg exactly once (inline function), includes string.h. Example app - LLMBenchmarkProvider: memBefore captured after unload; decode tok/s uses max(tokensUsed - 1, 0). - VLMBenchmarkProvider: memBefore after cleanup sequence; loadVLMModel moved inside do/catch so failures hit cleanup path. - RunAnywhereAIApp: MetalRT registration log reworded. - ToolSettingsView: NSExpression (crash-prone) replaced with SafeMathEvaluator recursive-descent parser; removed tool-input leak in debug print. Branch was first merged with latest origin/main (3 conflicts resolved: accepted upstream deletion of ArchiveUtility.swift — now tracked for C++ re-port; kept PR's UTF-8 boundary walk-back in llamacpp_backend with main's naming; kept both independent additions in llm_component). Deferred to follow-up issues: MetalRT build distribution unification, Swift-to-C++ archive streaming re-port.
1 parent 34adbed commit d707a0d

18 files changed

Lines changed: 364 additions & 96 deletions

File tree

‎Package.swift‎

Lines changed: 74 additions & 45 deletions
Original file line numberDiff line numberDiff line change
@@ -43,6 +43,15 @@ let useLocalBinaries = false // Toggle: true for local dev, false for release
4343
// Updated automatically by CI/CD during releases
4444
let sdkVersion = "0.19.7"
4545

46+
// MetalRT remote binary availability flag.
47+
// Set to `false` until a real checksum for RABackendMetalRT-v<sdkVersion>.zip
48+
// has been published. When `false`, the MetalRT product/targets are only
49+
// exposed under `useLocalBinaries = true`, so SPM resolution will not fail
50+
// for external consumers due to a placeholder checksum.
51+
let metalrtRemoteBinaryAvailable = false
52+
53+
let includeMetalRT = useLocalBinaries || metalrtRemoteBinaryAvailable
54+
4655
let package = Package(
4756
name: "runanywhere-sdks",
4857
platforms: [
@@ -82,14 +91,7 @@ let package = Package(
8291
targets: ["WhisperKitRuntime"]
8392
),
8493

85-
// =================================================================
86-
// MetalRT Backend - adds LLM/STT/TTS/VLM via custom Metal kernels
87-
// =================================================================
88-
.library(
89-
name: "RunAnywhereMetalRT",
90-
targets: ["MetalRTRuntime"]
91-
),
92-
],
94+
] + metalRTProducts(),
9395
dependencies: [
9496
.package(url: "https://github.com/apple/swift-crypto.git", from: "3.0.0"),
9597
.package(url: "https://github.com/Alamofire/Alamofire.git", from: "5.9.0"),
@@ -205,18 +207,63 @@ let package = Package(
205207
),
206208

207209
// =================================================================
208-
// MetalRT C Bridge Module - exposes rac_backend_metalrt_register()
210+
// WhisperKit Runtime Backend (Apple Neural Engine STT)
209211
// =================================================================
212+
.target(
213+
name: "WhisperKitRuntime",
214+
dependencies: [
215+
"RunAnywhere",
216+
.product(name: "WhisperKit", package: "whisperkit"),
217+
],
218+
path: "sdk/runanywhere-swift/Sources/WhisperKitRuntime",
219+
linkerSettings: [
220+
.linkedFramework("CoreML"),
221+
.linkedFramework("Accelerate"),
222+
]
223+
),
224+
225+
// =================================================================
226+
// RunAnywhere unit tests (e.g. AudioCaptureManager – Issue #198)
227+
// =================================================================
228+
.testTarget(
229+
name: "RunAnywhereTests",
230+
dependencies: ["RunAnywhere"],
231+
path: "sdk/runanywhere-swift/Tests/RunAnywhereTests"
232+
),
233+
234+
] + metalRTTargets() + binaryTargets()
235+
)
236+
237+
// =============================================================================
238+
// METALRT PRODUCT / TARGET GATING
239+
// =============================================================================
240+
// The RABackendMetalRT.xcframework is not yet published to GitHub releases
241+
// with a real checksum. To avoid SPM resolution failures for external
242+
// consumers due to a placeholder zero-checksum binary target, the MetalRT
243+
// product and its dependent targets are only included when:
244+
// - `useLocalBinaries == true` (local dev with a checked-out xcframework), or
245+
// - `metalrtRemoteBinaryAvailable == true` (once a real checksum is wired in).
246+
func metalRTProducts() -> [Product] {
247+
guard includeMetalRT else { return [] }
248+
return [
249+
.library(
250+
name: "RunAnywhereMetalRT",
251+
targets: ["MetalRTRuntime"]
252+
),
253+
]
254+
}
255+
256+
func metalRTTargets() -> [Target] {
257+
guard includeMetalRT else { return [] }
258+
return [
259+
// MetalRT C Bridge Module - exposes rac_backend_metalrt_register()
210260
.target(
211261
name: "MetalRTBackend",
212262
dependencies: ["RABackendMetalRTBinary"],
213263
path: "sdk/runanywhere-swift/Sources/MetalRTRuntime/include",
214264
publicHeadersPath: "."
215265
),
216-
217-
// =================================================================
218266
// MetalRT Runtime Backend (custom Metal GPU kernels)
219-
// =================================================================
220267
.target(
221268
name: "MetalRTRuntime",
222269
dependencies: [
@@ -237,34 +284,8 @@ let package = Package(
237284
.linkedFramework("ImageIO"),
238285
]
239286
),
240-
241-
// =================================================================
242-
// WhisperKit Runtime Backend (Apple Neural Engine STT)
243-
// =================================================================
244-
.target(
245-
name: "WhisperKitRuntime",
246-
dependencies: [
247-
"RunAnywhere",
248-
.product(name: "WhisperKit", package: "whisperkit"),
249-
],
250-
path: "sdk/runanywhere-swift/Sources/WhisperKitRuntime",
251-
linkerSettings: [
252-
.linkedFramework("CoreML"),
253-
.linkedFramework("Accelerate"),
254-
]
255-
),
256-
257-
// =================================================================
258-
// RunAnywhere unit tests (e.g. AudioCaptureManager – Issue #198)
259-
// =================================================================
260-
.testTarget(
261-
name: "RunAnywhereTests",
262-
dependencies: ["RunAnywhere"],
263-
path: "sdk/runanywhere-swift/Tests/RunAnywhereTests"
264-
),
265-
266-
] + binaryTargets()
267-
)
287+
]
288+
}
268289

269290
// =============================================================================
270291
// BINARY TARGET SELECTION
@@ -336,11 +357,6 @@ func binaryTargets() -> [Target] {
336357
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/RABackendONNX-v\(sdkVersion).zip",
337358
checksum: "809e2510da49f71f6d019e77bcc0a7e12e967f3b739ba0b9eea7adb77936edc0"
338359
),
339-
.binaryTarget(
340-
name: "RABackendMetalRTBinary",
341-
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/RABackendMetalRT-v\(sdkVersion).zip",
342-
checksum: "0000000000000000000000000000000000000000000000000000000000000000"
343-
),
344360
.binaryTarget(
345361
name: "ONNXRuntimeiOSBinary",
346362
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/onnxruntime-ios-v\(sdkVersion).zip",
@@ -353,6 +369,19 @@ func binaryTargets() -> [Target] {
353369
),
354370
]
355371

372+
// MetalRT remote binary is only appended once a real checksum has been
373+
// published. Until then the MetalRT product/targets are omitted from
374+
// the package graph entirely (see metalRTProducts/metalRTTargets).
375+
if metalrtRemoteBinaryAvailable {
376+
targets.append(
377+
.binaryTarget(
378+
name: "RABackendMetalRTBinary",
379+
url: "https://github.com/RunanywhereAI/runanywhere-sdks/releases/download/v\(sdkVersion)/RABackendMetalRT-v\(sdkVersion).zip",
380+
checksum: "0000000000000000000000000000000000000000000000000000000000000000" // TODO: replace with real checksum
381+
)
382+
)
383+
}
384+
356385
return targets
357386
}
358387
}

‎examples/ios/RunAnywhereAI/RunAnywhereAI/App/RunAnywhereAIApp.swift‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -491,7 +491,7 @@ struct RunAnywhereAIApp: App {
491491
)
492492
}
493493

494-
logger.info("✅ MetalRT models registered")
494+
logger.info("ℹ️ MetalRT runtime available; no downloadable MetalRT models are configured yet")
495495
#else
496496
logger.info("ℹ️ MetalRT not available (MetalRTRuntime not linked)")
497497
#endif

‎examples/ios/RunAnywhereAI/RunAnywhereAI/Features/Benchmarks/Services/LLMBenchmarkProvider.swift‎

Lines changed: 5 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -27,11 +27,11 @@ struct LLMBenchmarkProvider: BenchmarkScenarioProvider {
2727
let maxTokens = Int(scenario.parameters?["maxTokens"] ?? "") ?? 512
2828
var metrics = BenchmarkMetrics()
2929

30-
let memBefore = SyntheticInputGenerator.availableMemoryBytes()
31-
3230
// Ensure clean state: unload any model left over from Chat or a previous run
3331
try? await RunAnywhere.unloadModel()
3432

33+
let memBefore = SyntheticInputGenerator.availableMemoryBytes()
34+
3535
// Load
3636
let loadStart = Date()
3737
try await RunAnywhere.loadModel(model.id)
@@ -66,8 +66,9 @@ struct LLMBenchmarkProvider: BenchmarkScenarioProvider {
6666

6767
if let ttft = result.timeToFirstTokenMs, ttft > 0 {
6868
let decodeMs = e2eMs - ttft
69-
if decodeMs > 0, result.tokensUsed > 0 {
70-
metrics.decodeTokensPerSecond = Double(result.tokensUsed) / (decodeMs / 1000.0)
69+
let decodeTokens = max(result.tokensUsed - 1, 0)
70+
if decodeMs > 0, decodeTokens > 0 {
71+
metrics.decodeTokensPerSecond = Double(decodeTokens) / (decodeMs / 1000.0)
7172
}
7273
if result.inputTokens > 0 {
7374
metrics.prefillTokensPerSecond = Double(result.inputTokens) / (ttft / 1000.0)

‎examples/ios/RunAnywhereAI/RunAnywhereAI/Features/Benchmarks/Services/VLMBenchmarkProvider.swift‎

Lines changed: 6 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -28,21 +28,21 @@ struct VLMBenchmarkProvider: BenchmarkScenarioProvider {
2828
#if canImport(UIKit)
2929
var metrics = BenchmarkMetrics()
3030

31-
let memBefore = SyntheticInputGenerator.availableMemoryBytes()
32-
3331
// Ensure clean state: unload any VLM model left over from Camera or a previous run
3432
await RunAnywhere.unloadVLMModel()
3533
// Also unload any lingering LLM model to free memory headroom
3634
try? await RunAnywhere.unloadModel()
3735
// Brief pause to let iOS reclaim GPU/Metal memory from the previous model
3836
try await Task.sleep(nanoseconds: 500_000_000) // 0.5s
3937

40-
// Load
41-
let loadStart = Date()
42-
try await RunAnywhere.loadVLMModel(model)
43-
metrics.loadTimeMs = Date().timeIntervalSince(loadStart) * 1000
38+
let memBefore = SyntheticInputGenerator.availableMemoryBytes()
4439

4540
do {
41+
// Load
42+
let loadStart = Date()
43+
try await RunAnywhere.loadVLMModel(model)
44+
metrics.loadTimeMs = Date().timeIntervalSince(loadStart) * 1000
45+
4646
// Generate a small synthetic image inside an autoreleasepool so CoreGraphics
4747
// intermediates are released promptly before we allocate the vision encoder.
4848
let vlmImage: VLMImage = autoreleasepool {

‎examples/ios/RunAnywhereAI/RunAnywhereAI/Features/Settings/ToolSettingsView.swift‎

Lines changed: 130 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -100,7 +100,6 @@ class ToolSettingsViewModel: ObservableObject {
100100
}
101101
return "0"
102102
}()
103-
print("Calculator received args: \(args), using expression: '\(expression)'")
104103
// Clean the expression - remove any non-math characters
105104
let cleanedExpression = expression
106105
.replacingOccurrences(of: "=", with: "")
@@ -109,20 +108,12 @@ class ToolSettingsViewModel: ObservableObject {
109108
.replacingOccurrences(of: "÷", with: "/")
110109
.trimmingCharacters(in: .whitespacesAndNewlines)
111110

112-
// Validate expression contains only safe math characters
113-
let allowedChars = CharacterSet(charactersIn: "0123456789.+-*/() ")
114-
guard cleanedExpression.unicodeScalars.allSatisfy({ allowedChars.contains($0) }),
115-
!cleanedExpression.isEmpty else {
111+
// Safely evaluate using a deterministic parser that validates
112+
// grammar and returns errors instead of crashing (unlike
113+
// NSExpression, whose Obj-C exceptions cannot be caught from Swift).
114+
if let value = SafeMathEvaluator.evaluate(cleanedExpression) {
116115
return [
117-
"error": .string("Could not evaluate expression: \(expression)"),
118-
"expression": .string(expression)
119-
]
120-
}
121-
122-
let exp = NSExpression(format: cleanedExpression)
123-
if let result = exp.expressionValue(with: nil, context: nil) as? NSNumber {
124-
return [
125-
"result": .number(result.doubleValue),
116+
"result": .number(value),
126117
"expression": .string(expression)
127118
]
128119
}
@@ -416,6 +407,131 @@ enum WeatherService {
416407
}
417408
}
418409

410+
// MARK: - Safe Math Evaluator
411+
//
412+
// Deterministic recursive-descent parser for simple arithmetic expressions.
413+
// Replaces NSExpression(format:) which can raise uncaught Objective-C
414+
// exceptions (e.g. for "1 2", "(1+2", "1++2") that Swift's do-catch cannot
415+
// intercept. Supports the grammar:
416+
// expr := term (("+" | "-") term)*
417+
// term := factor (("*" | "/") factor)*
418+
// factor := ("+" | "-") factor | primary
419+
// primary := number | "(" expr ")"
420+
enum SafeMathEvaluator {
421+
static func evaluate(_ expression: String) -> Double? {
422+
var parser = Parser(input: expression)
423+
guard let value = parser.parseExpression() else { return nil }
424+
guard parser.isAtEnd else { return nil }
425+
guard value.isFinite else { return nil }
426+
return value
427+
}
428+
429+
private struct Parser {
430+
let scalars: [Character]
431+
var index: Int = 0
432+
433+
init(input: String) {
434+
self.scalars = Array(input)
435+
}
436+
437+
var isAtEnd: Bool {
438+
skipWhitespace()
439+
return index >= scalars.count
440+
}
441+
442+
mutating func skipWhitespace() {
443+
while index < scalars.count, scalars[index].isWhitespace {
444+
index += 1
445+
}
446+
}
447+
448+
mutating func peek() -> Character? {
449+
skipWhitespace()
450+
return index < scalars.count ? scalars[index] : nil
451+
}
452+
453+
mutating func advance() -> Character? {
454+
skipWhitespace()
455+
guard index < scalars.count else { return nil }
456+
let char = scalars[index]
457+
index += 1
458+
return char
459+
}
460+
461+
mutating func match(_ char: Character) -> Bool {
462+
if peek() == char {
463+
_ = advance()
464+
return true
465+
}
466+
return false
467+
}
468+
469+
mutating func parseExpression() -> Double? {
470+
guard var value = parseTerm() else { return nil }
471+
while let op = peek(), op == "+" || op == "-" {
472+
_ = advance()
473+
guard let rhs = parseTerm() else { return nil }
474+
value = op == "+" ? value + rhs : value - rhs
475+
}
476+
return value
477+
}
478+
479+
mutating func parseTerm() -> Double? {
480+
guard var value = parseFactor() else { return nil }
481+
while let op = peek(), op == "*" || op == "/" {
482+
_ = advance()
483+
guard let rhs = parseFactor() else { return nil }
484+
if op == "/" {
485+
guard rhs != 0 else { return nil }
486+
value /= rhs
487+
} else {
488+
value *= rhs
489+
}
490+
}
491+
return value
492+
}
493+
494+
mutating func parseFactor() -> Double? {
495+
if match("+") { return parseFactor() }
496+
if match("-") {
497+
guard let value = parseFactor() else { return nil }
498+
return -value
499+
}
500+
return parsePrimary()
501+
}
502+
503+
mutating func parsePrimary() -> Double? {
504+
guard let next = peek() else { return nil }
505+
if next == "(" {
506+
_ = advance()
507+
guard let value = parseExpression() else { return nil }
508+
guard match(")") else { return nil }
509+
return value
510+
}
511+
return parseNumber()
512+
}
513+
514+
mutating func parseNumber() -> Double? {
515+
skipWhitespace()
516+
let start = index
517+
var seenDot = false
518+
while index < scalars.count {
519+
let char = scalars[index]
520+
if char.isNumber {
521+
index += 1
522+
} else if char == "." && !seenDot {
523+
seenDot = true
524+
index += 1
525+
} else {
526+
break
527+
}
528+
}
529+
guard index > start else { return nil }
530+
return Double(String(scalars[start..<index]))
531+
}
532+
}
533+
}
534+
419535
#Preview {
420536
NavigationStack {
421537
Form {

0 commit comments

Comments
 (0)