Binary Data & Image Format

In this segment, you’ll learn how to convert Android Bitmap images to a format that Swift can process. You’ll create a custom file format for passing image data across the JNI boundary and implement the conversion logic in both directions.

Understanding image data representation is crucial for cross-language image processing. While Android uses Bitmap objects and Swift uses Data, both need to work with the same underlying pixel information.

Understanding RGBA Image Format

Digital images are composed of pixels, and each pixel contains color information. The RGBA format represents each pixel with four values:

• R (Red): 0-255
• G (Green): 0-255
• B (Blue): 0-255
• A (Alpha): 0-255 (transparency)

Each value takes 1 byte (8 bits), so each pixel requires 4 bytes total.

Example pixel values:

• Red: [255, 0, 0, 255] (fully opaque red)
• Green: [0, 255, 0, 255] (fully opaque green)
• Blue: [0, 0, 255, 255] (fully opaque blue)
• Semi-transparent white: [255, 255, 255, 128] (50% transparent)

For an image with dimensions 100x100 pixels, you need 100 * 100 * 4 = 40,000 bytes to store all pixel data.

Designing a Custom File Format

To pass image data between Kotlin and Swift, you’ll create a simple custom file format. While you could use standard formats like PNG or JPEG, a custom format is simpler for this educational purpose and demonstrates the data marshaling process clearly.

Your custom format consists of two parts:

Header (8 bytes):

• Bytes 0-3: Width as Int32 (big-endian)
• Bytes 4-7: Height as Int32 (big-endian)

Body (variable length):

• RGBA pixel data (width * height * 4 bytes)

Example: A 10x10 image requires 8 + (10 * 10 * 4) = 408 bytes total.

Big-endian means the most significant byte comes first. For example, the number 1024 (0x00000400) is stored as [0x00, 0x00, 0x04, 0x00].

Creating the Image Processing Repository

Now create a repository to handle image format conversion and Swift filter calls.

Create a new file app/src/main/java/com/kodeco/android/swiftsdkforandroid/taskmanager/repository/ImageProcessingRepository.kt:

package com.kodeco.android.swiftsdkforandroid.taskmanager.repository

import android.content.Context
import android.graphics.Bitmap
import com.kodeco.android.taskmanagerkit.ImageProcessor
import java.io.File
import java.io.FileOutputStream

// 1
object ImageProcessingRepository {

  // 2
  enum class FilterType(val swiftName: String) {
    GRAYSCALE("grayscale"),
    BLUR("blur"),
    BRIGHTER("brighter"),
    DARKER("darker")
  }

  // 3
  fun applyFilter(
    bitmap: Bitmap,
    filterType: FilterType,
    context: Context,
    amount: Float = 0f
  ): Bitmap? {
    try {
      // 4
      val scaledBitmap = downscaleForProcessing(bitmap, maxDimension = 1024)

      val width = scaledBitmap.width
      val height = scaledBitmap.height

      // 5
      val inputFile = File(context.cacheDir, "filter_input_${System.currentTimeMillis()}.rgba")
      FileOutputStream(inputFile).use { out ->
        // 6
        out.write(width shr 24)
        out.write(width shr 16)
        out.write(width shr 8)
        out.write(width)
        out.write(height shr 24)
        out.write(height shr 16)
        out.write(height shr 8)
        out.write(height)

        // 7
        val pixels = IntArray(width * height)
        scaledBitmap.getPixels(pixels, 0, width, 0, 0, width, height)
        for (pixel in pixels) {
          out.write((pixel shr 16) and 0xFF) // R
          out.write((pixel shr 8) and 0xFF)  // G
          out.write(pixel and 0xFF)          // B
          out.write((pixel shr 24) and 0xFF) // A
        }
      }

      // 8
      val outputFile = File(context.cacheDir, "filter_output_${System.currentTimeMillis()}.rgba")

      // 9
      val success = ImageProcessor.processImageFile(
        inputFile.absolutePath,
        outputFile.absolutePath,
        filterType.swiftName,
        amount
      )

      // 10
      val result = if (success && outputFile.exists()) {
        val bytes = outputFile.readBytes()
        if (bytes.size >= 8) {
          // 11
          val w = ((bytes[0].toInt() and 0xFF) shl 24) or
                  ((bytes[1].toInt() and 0xFF) shl 16) or
                  ((bytes[2].toInt() and 0xFF) shl 8) or
                  (bytes[3].toInt() and 0xFF)
          val h = ((bytes[4].toInt() and 0xFF) shl 24) or
                  ((bytes[5].toInt() and 0xFF) shl 16) or
                  ((bytes[6].toInt() and 0xFF) shl 8) or
                  (bytes[7].toInt() and 0xFF)

          // 12
          val pixelData = IntArray(w * h)
          var byteIndex = 8
          for (i in pixelData.indices) {
            val r = bytes[byteIndex++].toInt() and 0xFF
            val g = bytes[byteIndex++].toInt() and 0xFF
            val b = bytes[byteIndex++].toInt() and 0xFF
            val a = bytes[byteIndex++].toInt() and 0xFF
            pixelData[i] = (a shl 24) or (r shl 16) or (g shl 8) or b
          }

          val resultBitmap = Bitmap.createBitmap(w, h, Bitmap.Config.ARGB_8888)
          resultBitmap.setPixels(pixelData, 0, w, 0, 0, w, h)
          resultBitmap
        } else {
          null
        }
      } else {
        null
      }

      // 13
      inputFile.delete()
      outputFile.delete()

      return result
    } catch (e: Exception) {
      e.printStackTrace()
      return null
    }
  }

  // 14
  private fun downscaleForProcessing(bitmap: Bitmap, maxDimension: Int): Bitmap {
    val width = bitmap.width
    val height = bitmap.height

    // 15
    if (width <= maxDimension && height <= maxDimension) {
      return bitmap
    }

    // 16
    val scale = if (width > height) {
      maxDimension.toFloat() / width
    } else {
      maxDimension.toFloat() / height
    }

    val newWidth = (width * scale).toInt()
    val newHeight = (height * scale).toInt()

    return Bitmap.createScaledBitmap(bitmap, newWidth, newHeight, true)
  }
}

What each section does:

1. Singleton object: Single instance handles all image processing.
2. FilterType enum: Maps Kotlin filter names to Swift filter names.
3. applyFilter function: Main entry point for applying filters.
4. downscaleForProcessing: Limits image size to 1024x1024 for performance.
5. Create input file: Temporary file in cache directory for passing data to Swift.
6. Write header: Encodes width and height as 4 bytes each (big-endian).
7. Write pixel data: Converts Bitmap pixels to RGBA bytes.
8. Create output file: Temporary file where Swift writes the processed image.
9. Call Swift: Invokes Swift ImageProcessor through JNI bindings.
10. Check result: Verifies Swift processing succeeded.
11. Parse header: Reads width and height from the output file.
12. Parse pixels: Converts RGBA bytes back to Android pixel format (ARGB).
13. Cleanup: Deletes temporary files.
14. downscaleForProcessing helper: Reduces image size if needed.
15. Check if scaling needed: Returns original if already small enough.
16. Calculate scale: Maintains aspect ratio while fitting in max dimension.

Build and run the app now to ensure the ImageProcessingRepository compiles correctly before continuing.

Understanding Bit Shifting for Header Encoding

The header encoding uses bit shift operations to convert a 32-bit integer into 4 bytes.

Writing the header (Kotlin):

val width = 1024  // 0x00000400 in hexadecimal
out.write(width shr 24)  // Writes 0x00 (byte 0)
out.write(width shr 16)  // Writes 0x00 (byte 1)
out.write(width shr 8)   // Writes 0x04 (byte 2)
out.write(width)         // Writes 0x00 (byte 3)

The shr operator shifts bits right, extracting each byte. The and 0xFF operation ensures only the bottom 8 bits remain.

Reading the header (Swift):

Swift reads these same 4 bytes and reconstructs the original integer by shifting left and combining with bitwise OR:

let width = Int32((UInt32(bytes[0]) << 24) |
                  (UInt32(bytes[1]) << 16) |
                  (UInt32(bytes[2]) << 8) |
                  UInt32(bytes[3]))

This ensures both languages interpret the dimension values identically.

Understanding Pixel Format Conversion

Android stores pixels as ARGB in a packed Int, but you need to convert them to separate RGBA bytes for the file format.

Android pixel format (32-bit Int):

Bit position: 31-24  23-16  15-8   7-0
Value:        Alpha  Red    Green  Blue

File format (4 bytes per pixel):

Byte 0: Red
Byte 1: Green
Byte 2: Blue
Byte 3: Alpha

Extraction with bit shifts:

val pixel = pixels[i]  // Example: 0xFF800000 (red, fully opaque)
val r = (pixel shr 16) and 0xFF  // Extracts 0x80 (128)
val g = (pixel shr 8) and 0xFF   // Extracts 0x00 (0)
val b = pixel and 0xFF           // Extracts 0x00 (0)
val a = (pixel shr 24) and 0xFF  // Extracts 0xFF (255)

The reverse operation when reading back combines the bytes:

val pixel = (a shl 24) or (r shl 16) or (g shl 8) or b

Understanding the Complete Data Flow

Here’s how image data flows between Kotlin and Swift:

1. User captures photo, Kotlin receives Bitmap.
2. Kotlin downscales Bitmap if too large.
3. Kotlin writes custom format file (header + RGBA pixels).
4. Kotlin calls Swift ImageProcessor.processImageFile().
5. Swift reads file, parses header, extracts pixel data.
6. Swift applies filter algorithm to pixels.
7. Swift writes filtered pixels with header to output file.
8. Kotlin reads output file, parses header and pixels, creates Bitmap.
9. Kotlin displays filtered Bitmap in UI.

This roundtrip demonstrates bidirectional data flow where both languages work with the same underlying data but use their own native types (Bitmap vs Data).

Key Takeaways

In this segment, you’ve:

• Learned how RGBA format represents pixel data.
• Designed a custom file format with header and pixel data.
• Implemented ImageProcessingRepository for format conversion.
• Understood bit shifting for integer encoding.
• Learned pixel format differences between Android and your custom format.
• Understood the complete data flow from Kotlin to Swift and back.

Your image data can now flow between Kotlin and Swift. In the next segment, you’ll implement the actual filter algorithms in Swift that process this pixel data.