chore: rename modules to have logic names and also lower name conflicts

Author: AleksandrSl

cmp/compiler/README.md

@@ -102,7 +102,6 @@ The compiler is organized into several key modules:
 
 - **`src/index.ts`** - Main package exports (plugins, types)
 - **`src/types.ts`** - Core TypeScript types
-- **`src/dev-config.ts`** - Development configuration utilities
 
 ## Contributing
 

cmp/compiler/Requirements.md

@@ -1,34 +1,50 @@
-# Translation compiler requirements
+## Requirements
 
-## Overview of the process
+1. The core thing is DX. We want compiler to work as seamless as possible.
+2. It should work at least with turbopack (since it's next default bundler), webpack and vite.
+3. HMR should work, whenever we update the code the state of the app should be kept as it owuld be without the compiler.
+4. There should be a way to override the translations
+5. We should be able to translate the attributes, the most obvious ones are alt and aria-label, aria-description and
+   such, but we may also add a way to translate other string attributes too.
+6. We should keep translations calls to the minimum.
+7. We start with react, but it would be nice to apply the same approach to other frameworks. Like vue and astro. So the
+   system should be flexible.
+8. The translations should be possible to commit in the repo, so we can version them. Maybe we can also consider
+   conforming to the existing standard?
 
-### Dev mode
+## Overall architecture
 
-- When the app is started we should extract translatable text from the code and inject our translation functions.
-- When the user changes the code, we should re-extract the text and update the translation functions.
-- If a non source language is used (e.g. sources are in english, and user switches to the german) we want to start translations and update the app when they are ready.
+Given that we still may want translations in the dev mode, but we want to keep them to minimum and do not interfere with
+the development flow, we may consider the following approach:
 
-### Build mode
+1. Do not translate the text unless explicitly requested.
+2. Allow using pseudolocalization for even more simple checks. It shows which part of the UI can be translated by
+   compiler, and also using varying width of the text highlights part of the interface not ready for varying languages.
 
-- During build we need to extract all the translatable text from the code and inject our translation functions.
-- While the build is in progress, we should perform the translation and create files with translations.
-- When the build ends we should make sure that all the translations are in place.
-- Ideally split translations into multiple chunks, according to the js chunks maybe.
+This leads us to the question, how to lazily request the translations in the dev mode.
+Let's work with the SPA apps as an example (since SSR apps may have more options).
+Both currently rendered components and the locale are client side.
+Cache for translations is stored in the filesystem in the project.
+So the way to receive a request from the browser to the host that will get the translations and store the cache.
 
-In the end we should have a built app with all the translations in place.
+The most straightforward way to do this is to use the usual web server. App will send a request for translations when needed,
+server components can do the same. This adds a bit of latency but keeps the code uniform.
 
-## Problems of the current implementation
+With this approach we get couple more questions:
 
-We want to avoid fetching translations if they are not needed. To achieve this, we need to ask for translations if they are missing.
-We also want to cache the translations to avoid unnecessary requests, by default, the cache is on the user machine.
-But client side components in the browser can't access the user machine to save the cache.
+- How to start the server
+- How to make the client aware about server's address
+- How to batch the calls, both client and server side.
 
-This can be solved by adding separate route/middleware on the user machine that can be called from the client and will serve the translations.
-We can use Vite middleware and Next routeHandler to do this.
+How to start the server
+While it sounds simple, it's not that simple if you don't want to create any external scripts.
+We have at least three environments:
 
-- The approaches differ for different bundlers
-- We do not want to ask the user to add the extra route handlers to their app.
+- Next turbopack: no startup hooks.
+- Next webpack: there is compiler.hooks.watchRun. Well, watchRun runs on every script compilation during hot reload.
+- Vite: buildStart.
 
-## Possible solution
+So next with turbopack is the most complex case.
 
-Use a separate server that will serve the translations. It can be started by the compiler. Then client components can fetch the translations from the server.
+It shows which part of the UI can be translated by
+compiler - can be done easier? Add some colors

cmp/compiler/package.json

@@ -29,20 +29,20 @@
         "default": "./build/react/client/index.cjs"
       }
     },
-    "./dev-config": {
-      "types": "./build/dev-config.d.ts",
-      "import": "./build/dev-config.mjs",
-      "require": "./build/dev-config.cjs"
+    "./virtual/config": {
+      "types": "./build/virtual/config.d.ts",
+      "import": "./build/virtual/config.mjs",
+      "require": "./build/virtual/config.cjs"
     },
-    "./locale/server": {
-      "types": "./build/locale/server.d.ts",
-      "import": "./build/locale/server.mjs",
-      "require": "./build/locale/server.cjs"
+    "./virtual/locale/server": {
+      "types": "./build/virtual/locale/server.d.ts",
+      "import": "./build/virtual/locale/server.mjs",
+      "require": "./build/virtual/locale/server.cjs"
     },
-    "./locale/client": {
-      "types": "./build/locale/client.d.ts",
-      "import": "./build/locale/client.mjs",
-      "require": "./build/locale/client.cjs"
+    "./virtual/locale/client": {
+      "types": "./build/virtual/locale/client.d.ts",
+      "import": "./build/virtual/locale/client.mjs",
+      "require": "./build/virtual/locale/client.cjs"
     },
     "./react/client": {
       "types": "./build/react/index.d.ts",
@@ -91,10 +91,10 @@
       "import": "./build/plugin/next-compiler-loader.mjs",
       "require": "./build/plugin/next-compiler-loader.cjs"
     },
-    "./next-dev-config-loader": {
-      "types": "./build/plugin/next-dev-config-loader.d.ts",
-      "import": "./build/plugin/next-dev-config-loader.mjs",
-      "require": "./build/plugin/next-dev-config-loader.cjs"
+    "./next-config-loader": {
+      "types": "./build/plugin/next-config-loader.d.ts",
+      "import": "./build/plugin/next-config-loader.mjs",
+      "require": "./build/plugin/next-config-loader.cjs"
     },
     "./next-locale-server-loader": {
       "types": "./build/plugin/next-locale-server-loader.d.ts",

cmp/compiler/src/plugin/next-config-loader.ts

@@ -0,0 +1,11 @@
+import type { LingoConfig } from "../types";
+import { generateConfigModule } from "../virtual/code-generator";
+
+/**
+ * Loader for config module
+ * Generates full module code - ignores source file (template is only for types)
+ */
+export default function nextConfigLoader(this: any, _source: string): string {
+  const config: LingoConfig = this.getOptions();
+  return generateConfigModule(config);
+}

cmp/compiler/src/plugin/next-dev-config-loader.ts

@@ -4,7 +4,7 @@
  */
 
 import type { LingoConfig } from "../types";
-import { generateClientLocaleModule } from "./virtual-modules-code-generator";
+import { generateClientLocaleModule } from "../virtual/code-generator";
 
 export default function nextLocaleClientLoader(
   this: any,

cmp/compiler/src/plugin/next-locale-client-loader.ts

@@ -4,7 +4,7 @@
  */
 
 import type { LingoConfig } from "../types";
-import { generateServerLocaleModule } from "./virtual-modules-code-generator";
+import { generateServerLocaleModule } from "../virtual/code-generator";
 
 export default function nextLocaleServerLoader(
   this: any,

cmp/compiler/src/plugin/next-locale-server-loader.ts

@@ -42,8 +42,8 @@ export function loaders({
     },
   };
 
-  const devConfigLoader = {
-    loader: "@lingo.dev/compiler/next-dev-config-loader",
+  const configLoader = {
+    loader: "@lingo.dev/compiler/next-config-loader",
     options: {
       ...common,
       dev: {
@@ -90,43 +90,43 @@ export function loaders({
     devConfig: translationServerUrl
       ? {
           turbopack: {
-            pattern: "**/dev-config.mjs",
+            pattern: "**/virtual/config.mjs",
             config: {
-              loaders: [devConfigLoader],
+              loaders: [configLoader],
             },
           },
           webpack: {
             enforce: "pre",
-            test: /dev-config\.mjs$/i,
-            use: [devConfigLoader],
+            test: /virtual\/config\.mjs$/i,
+            use: [configLoader],
           },
         }
       : {},
 
     localeServer: {
       turbopack: {
-        pattern: "**/locale/server.mjs",
+        pattern: "**/virtual/locale/server.mjs",
         config: {
           loaders: [localeServerLoader],
         },
       },
       webpack: {
         enforce: "pre",
-        test: /locale[\\/]server\.mjs$/i,
+        test: /virtual\/locale[\\/]server\.mjs$/i,
         use: [localeServerLoader],
       },
     },
 
     localeClient: {
       turbopack: {
-        pattern: "**/locale/client.mjs",
+        pattern: "**/virtual/locale/client.mjs",
         config: {
           loaders: [localeClientLoader],
         },
       },
       webpack: {
         enforce: "pre",
-        test: /locale[\\/]client\.mjs$/i,
+        test: /virtual\/locale[\\/]client\.mjs$/i,
         use: [localeClientLoader],
       },
     },

cmp/compiler/src/plugin/next.ts

@@ -19,9 +19,9 @@ import { logger } from "../utils/logger";
 import { useI18nRegex } from "./transform/use-i18n";
 import {
   generateClientLocaleModule,
-  generateDevConfigModule,
+  generateConfigModule,
   generateServerLocaleModule,
-} from "./virtual-modules-code-generator";
+} from "../virtual/code-generator";
 import { processBuildTranslations } from "./build-translator";
 import { registerCleanupOnCurrentProcess } from "./cleanup";
 import path from "path";
@@ -52,17 +52,17 @@ function tryLocalOrReturnVirtual(
  * If customFileCheck is defined, the specified file will be first searched for, and if not found virtual module will be used.
  */
 const virtualModules = {
-  "@lingo.dev/compiler/dev-config": {
-    virtualId: "\0virtual:lingo-dev-config",
-    loader: (config: LingoConfig) => generateDevConfigModule(config),
+  "@lingo.dev/compiler/virtual/config": {
+    virtualId: "\0virtual:lingo-config",
+    loader: (config: LingoConfig) => generateConfigModule(config),
     customFileCheck: undefined,
   },
-  "@lingo.dev/compiler/locale/server": {
+  "@lingo.dev/compiler/virtual/locale/server": {
     virtualId: "\0virtual:locale-resolver.server" as const,
     loader: (config: LingoConfig) => generateServerLocaleModule(config),
     customFileCheck: "locale-resolver.server.ts" as const,
   },
-  "@lingo.dev/compiler/locale/client": {
+  "@lingo.dev/compiler/virtual/locale/client": {
     virtualId: "\0virtual:locale-resolver.client" as const,
     loader: (config: LingoConfig) => generateClientLocaleModule(config),
     customFileCheck: "locale-resolver.client.ts" as const,
@@ -110,6 +110,27 @@ export const lingoUnplugin = createUnplugin<
     );
   };
 
+  async function startServer() {
+    const server = await startTranslationServer({
+      startPort,
+      onError: (err) => {
+        logger.error("Translation server error:", err);
+      },
+      onReady: (port) => {
+        logger.info(`Translation server started successfully on port: ${port}`);
+      },
+      config,
+    });
+    // I don't like this quite a lot. But starting server inside the loader seems lame.
+    config.dev.translationServerUrl = server.getUrl();
+    registerCleanupOnCurrentProcess({
+      asyncCleanup: async () => {
+        await translationServer.stop();
+      },
+    });
+    return server;
+  }
+
   return {
     name: PLUGIN_NAME,
     enforce: "pre", // Run before other plugins (especially before React plugin)
@@ -119,29 +140,12 @@ export const lingoUnplugin = createUnplugin<
         const metadataFilePath = getMetadataPath();
 
         cleanupExistingMetadata(metadataFilePath);
-
         registerCleanupOnCurrentProcess({
           cleanup: () => cleanupExistingMetadata(metadataFilePath),
         });
-        if (isDev && !translationServer) {
-          translationServer = await startTranslationServer({
-            startPort,
-            onError: (err) => {
-              logger.error("Translation server error:", err);
-            },
-            onReady: (port) => {
-              logger.info(
-                `Translation server started successfully on port: ${port}`,
-              );
-            },
-            config,
-          });
 
-          registerCleanupOnCurrentProcess({
-            asyncCleanup: async () => {
-              await translationServer.stop();
-            },
-          });
+        if (isDev && !translationServer) {
+          translationServer = await startServer();
         }
       },
 
@@ -177,23 +181,7 @@ export const lingoUnplugin = createUnplugin<
 
       compiler.hooks.watchRun.tapPromise(PLUGIN_NAME, async () => {
         if (webpackMode === "development" && !translationServer) {
-          translationServer = await startTranslationServer({
-            startPort,
-            onError: (err) => {
-              logger.error("Translation server error:", err);
-            },
-            onReady: (port) => {
-              logger.info(
-                `Translation server started successfully on port: ${port}`,
-              );
-            },
-            config,
-          });
-          registerCleanupOnCurrentProcess({
-            asyncCleanup: async () => {
-              await translationServer.stop();
-            },
-          });
+          translationServer = await startServer();
         }
       });
 

cmp/compiler/src/plugin/unplugin.ts

@@ -16,7 +16,7 @@ import {
 import type { ReactNode } from "react";
 import { logger } from "../../utils/logger";
 // Keep this import full for replacement during build.
-import { getServerLocale } from "@lingo.dev/compiler/locale/server";
+import { getServerLocale } from "@lingo.dev/compiler/virtual/locale/server";
 import type { LocaleCode } from "lingo.dev/spec";
 
 /**