Add support for using kotlinx-serialization rather than Jackson

[Luna712]

If accepted, I will slowly migrate the app and library to kotlinx-serialization, standardizing all the different JSON parsers we use as well. kotlinx-serialization is fully KMP and also more performant, because unlike Jackson, it doesn't use runtime reflection, it uses validation at compiler time.

[fire-light43]

A good start, but I want more insurance that this works when we migrate. Therefore I have created a comprehensive test to validate the move, please see if something like this can be included in androidTest in this pull request.

package com.lagradost.cloudstream3

import androidx.test.ext.junit.runners.AndroidJUnit4
import androidx.test.platform.app.InstrumentationRegistry
import com.lagradost.cloudstream3.utils.AppUtils.toJson
import dalvik.system.DexFile
import kotlinx.serialization.ExperimentalSerializationApi
import kotlinx.serialization.InternalSerializationApi
import kotlinx.serialization.KSerializer
import kotlinx.serialization.Serializable
import kotlinx.serialization.json.Json
import kotlinx.serialization.serializer
import kotlinx.serialization.serializerOrNull
import org.instancio.Instancio
import org.junit.Test
import org.junit.runner.RunWith
import kotlin.reflect.KClass
import kotlin.reflect.jvm.jvmName
import kotlin.test.assertEquals
import kotlin.test.assertNotNull

@RunWith(AndroidJUnit4::class)
class SerializationClassTester {
    // Same as app, or using app reference
    val jacksonMapper = mapper
    val kotlinxMapper = Json {
        ignoreUnknownKeys = true
    }

    @Test
    fun isIdenticalSerialization() {
        val serializableClasses = findSerializableClasses("com.lagradost")
        println("Number of serializable classes: ${serializableClasses.size}")

        serializableClasses.forEach { kClass ->
            val instance = Instancio.create(kClass.java)

            val jacksonJson = jacksonMapper.writeValueAsString(instance)
            val kotlinxJson = serializeWithKotlinx(kClass, instance)

            assertEquals(
                jacksonJson,
                kotlinxJson,
                """
                    Serialization mismatch for:
                    ${kClass.qualifiedName}

                    Jackson:
                    $jacksonJson

                    Kotlinx:
                    $kotlinxJson
                    
                    """.trimIndent()
            )
            println("Identical serialization for: ${kClass.jvmName}")
        }
    }

    @OptIn(InternalSerializationApi::class, ExperimentalSerializationApi::class)
    @Test
    fun isIdenticalDeserialization() {
        val serializableClasses = findSerializableClasses("com.lagradost")
        println("Number of serializable classes: ${serializableClasses.size}")

        serializableClasses.forEach { kClass ->
            val instance = Instancio.create(kClass.java)
            // Convert to JSON to get example JSON object
            // We prefer jackson here because the app may have many jackson JSON strings in local storage
            val originalJson = jacksonMapper.writeValueAsString(instance)

            // Create an object from the JSON using kotlinx
            val serializer =
                kClass.serializerOrNull() ?: kotlinxMapper.serializersModule.getContextual(kClass)
            assertNotNull(serializer, "The class: ${kClass.jvmName} must be serializable!")
            val kotlinxDecoded = kotlinxMapper.decodeFromString(serializer, originalJson)

            // Create an object from the JSON using jackson
            val mapperDecoded = jacksonMapper.readValue(originalJson, kClass.java)


            // Deep inspect both object using the mapper toJson function.
            // This deep equality check can be performed using other methods, but this just works.
            val jacksonJson = mapperDecoded.toJson()
            val kotlinxJson = kotlinxDecoded.toJson()

            assertEquals(
                jacksonJson,
                kotlinxJson,
                """
                    Serialization mismatch for:
                    ${kClass.qualifiedName}

                    Jackson:
                    $jacksonJson

                    Kotlinx:
                    $kotlinxJson
                    
                    """.trimIndent()
            )
            println("Identical deserialization for: ${kClass.jvmName}")
        }
    }

    private fun findSerializableClasses(packageName: String): List<KClass<*>> {
        val context = InstrumentationRegistry
            .getInstrumentation()
            .targetContext

        val dexFile = DexFile(context.packageCodePath)

        return dexFile.entries()
            .toList()
            .filter { it.startsWith(packageName) }
            .mapNotNull {
                runCatching { Class.forName(it).kotlin }
                    .onFailure { e ->
                        // Log.e("DEX", "Failed loading $it", e)
                    }
                    .getOrNull()
            }.filter { kClass ->
                kClass.java.annotations.any {
                    it is Serializable
                }
            }
    }

    @OptIn(InternalSerializationApi::class)
    @Suppress("UNCHECKED_CAST")
    private fun serializeWithKotlinx(
        kClass: KClass<*>,
        value: Any
    ): String {
        val serializer = kClass.serializer() as KSerializer<Any>
        return kotlinxMapper.encodeToString(serializer, value)
    }
}

I use

    androidTestImplementation(kotlin("test"))
    androidTestImplementation("org.instancio:instancio-core:5.5.1")
    androidTestImplementation(libs.kotlinx.serialization.json)

[Luna712]

The two suggestions are done. I also preemptively fixed issues that could have been caused during migration including bugs that broke backup and other stuff. I also added support to the rest of the app by using helper methods rather than mapper directly etc... I will add the test later also. For the record, for testing this I literally converted every single aspect of the app to kotlinx serialization and nothing broke now, but I will do smaller PRs later to convert sections of the app incrementally to make things a little easier to test later on.

[Luna712]

@fire-light43 I added the test now with some minor changes to what you had above to fix some deprecation warnings and a couple other things. I did #2792 so these tests run when making PRs also, since I plan to do those to start migrating a lot of things I thought that would make things easier also.

[fire-light43]

Looks good, just some minor things. It looks ready to merge very soon.

Good choice going with ClassGraph 👍

[fire-light43]

This logic is now replicated 3 times in different files. We should have one singular internal implementation all other methods use.

[Luna712]

Yeah I thought about that but it needed done a little differently here so I just replicated it. I will see about combining it though. I was mostly just copying how mapper was done before which we had it copied way to many times as well I think.

[fire-light43]

Shouldn't we register this serializer globally?

[Luna712]

Would rather not. We only need it in two places. But if you think we should no issues from me.

[fire-light43]

Do what you think is best, but an explanatory comment might be good if we limit the usage of this.

[Luna712]

Absolutely fair. I will add a comment. Very easy to use though for example:

    @Serializable
    data class MinimalVideoLink(
        @SerialName("uri")
        @Serializable(with = UriSerializer::class)
        val uri: Uri?,
        ...
    )

is one of the only two places we will need it. I will describe this in a comment though.

[fire-light43]

Probably good to have a @Prerelease annotation here

[Luna712]

Good idea, thanks for that.

[fire-light43]

I think also using encodeDefaults = true better matches our current jackson

[Luna712]

Good idea, thanks for that.

[Luna712]

I will do changes tomorrow more than likely rather than tonight. Thanks for the review again!

[fire-light43]

No problem, this is already very fast progress