JSON Import

Detail modul tryout_json_import.py - import snapshot JSON dari Sejoli Tryout, diff detection, variant stale.

Modul: JSON Import

Source: backend/app/services/tryout_json_import.py Router: backend/app/routers/import_export.py

⭐ Alur import utama klien — tryout dibuat di Sejoli Tryout, di-export ke JSON, lalu di-import ke app.

Tanggung Jawab

Import snapshot JSON yang di-export plugin Sejoli Tryout ke database app. Modul ini:

Parse struktur JSON export Sejoli (multi-tryout support)
Deteksi diff (new / updated / unchanged / removed) terhadap data existing
Upsert soal dengan content hash untuk hindari update tidak perlu
Manage variant stale (AI variants yang basisnya berubah)
Tracking via snapshot table untuk audit

Sumber Data

JSON export dari plugin Sejoli Tryout. Struktur umum:

json
{
  "export_info": {
    "exported_at": "2026-06-15 22:29:44",
    "tryout_id": 132380,
    "exported_by": "mtheyellowjacket"
  },
  "tryouts": {
    "tryout_132380": {
      "info": {
        "id": 132380,
        "title": "Tryout PEKA Logika Paket 1",
        "status": "publish",
        "duration": "40",
        "instruction": "...",
        "passing_grade": "60",
        "max_attempt": "1",
        "product_ids": ["132499", "132557", ...],
        "permalink": "https://member.theyellowjacket.id/..."
      },
      "questions": [
        {
          "id": 84,
          "category_id": 8,
          "category_name": "Simpulan Logis",
          "title": "Jika bencana kekeringan...",
          "question": "<p>...</p>",
          "options": [
            {"increment": "A", "text": "..."},
            {"increment": "B", "text": "..."}
          ],
          "answer": "A",
          "explanation": "..."
        }
      ]
    }
  }
}

Workflow Import

flowchart TD
    A["Admin upload JSON"] --> B["POST /import-export/tryout-json/preview"]
    B --> C["Parse + validate struktur"]
    C --> D["Compute content hash tiap soal"]
    D --> E["Diff dengan Item existing di App DB"]
    E --> F["Return TryoutPreview: new, updated, unchanged, removed"]
    F --> G{"Admin review diff"}
    G -->|"cancel"| H["Stop, nothing committed"]
    G -->|"approve"| I["POST /import-export/tryout-json commit"]
    I --> J["Upsert Item ke DB"]
    J --> K["Save TryoutImportSnapshot untuk audit"]
    J --> L["Mark variant stale jika basis berubah"]
    K --> M["Return summary: inserted, updated, skipped"]
    L --> M

Content Hash (SHA-256)

Untuk deteksi perubahan konten soal, modul pakai SHA-256 dari field-field kunci:

python
def snapshot_question_content_hash(question: dict) -> str:
    """Hash field: question, explanation, correct_answer, options."""
    return _sha256({
        "question": _normalize_whitespace(question.get("question")),
        "explanation": _normalize_whitespace(question.get("explanation")) or None,
        "correct_answer": _normalize_whitespace(question.get("answer")).upper(),
        "options": _normalize_options(question.get("options") or []),
    })

Tujuan:

Skip update kalau konten tidak berubah (hemat I/O)
Deteksi soal yang diedit di Sejoli (hash beda)
Tracking perubahan untuk audit log

Yang di-hash (bukan seluruh JSON):

question (stem) — setelah normalize whitespace
explanation
correct_answer — di-uppercase
options — list of {increment, text} setelah normalize

Yang tidak di-hash: id, category_id, category_name (metadata, bukan konten).

Diff Detection

Fungsi find_tryout_payload + _normalize_question menghasilkan summary diff:

python
@dataclass
class QuestionDiffSummary:
    total_questions: int       # total soal di JSON
    new_questions: int         # belum ada di App DB
    updated_questions: int     # ada tapi hash berubah
    unchanged_questions: int   # ada dan hash sama (akan di-skip)
    removed_questions: int     # ada di App DB tapi tidak di JSON
    missing_option_labels: int # soal tanpa label option (A, B, C, D)

Preview (sebelum commit) return:

python
@dataclass
class TryoutPreview:
    source_tryout_id: str      # dari JSON
    source_key: str            # mis. "tryout_132380"
    title: str
    permalink: str | None
    question_diff: QuestionDiffSummary
    warnings: list[str]        # mis. "3 questions missing option labels"

Admin lihat preview dulu, baru decide commit atau cancel.

Variant Stale Handling

Kalau soal basis berubah (hash berbeda), semua AI variants yang di-generate dari basis itu harus di-tandai stale (tidak layak dipakai lagi):

python
def mark_item_variants_stale(item: Item) -> None:
    for variant in item.variants or []:
        status = variant.variant_status or "draft"
        if status.startswith(STALE_PREFIX):
            continue   # sudah stale, skip
        if status in VARIANT_REVIEW_STATUSES:
            # mis. "approved" → "stale:approved" (preserve original)
            variant.variant_status = f"{STALE_PREFIX}{status}"
        else:
            variant.variant_status = "stale"

Kenapa penting?

AI variant dibuat berdasarkan konten basis
Kalau basis berubah (mis. typo diperbaiki, opsi diganti), variant mungkin tidak relevan lagi
Tapi tidak dihapus — bisa di-restore kalau basis ternyata balik

python
def restore_item_variants(item: Item) -> None:
    """Restore variant yang status-nya diawali STALE_PREFIX."""
    for variant in item.variants or []:
        status = variant.variant_status or ""
        if status.startswith(STALE_PREFIX):
            # "stale:approved" → "approved"
            restored = status[len(STALE_PREFIX):] or "draft"
            variant.variant_status = restored

Snapshot Audit

Setiap import commit, snapshot disimpan di tabel tryout_import_snapshots:

Field	Arti
`id`	Primary key
`website_id`	Tenant
`tryout_id`	Tryout target
`source_tryout_id`	ID dari Sejoli
`raw_payload`	JSON mentah yang di-import
`imported_at`	Timestamp
`summary`	`{inserted, updated, skipped, errors}`

Tujuan:

Audit trail — siapa import apa, kapan
Reproducibility — bisa replay import dari snapshot
Rollback — bisa compare snapshot lama vs baru untuk undo

Fungsi Publik

Fungsi	Tujuan
`find_tryout_payload(payload, target_id)`	Extract tryout tertentu dari JSON multi-tryout
`normalize_snapshot_rows(tryout_payload)`	Normalize list questions jadi rows siap-import
`snapshot_options_to_item_options(raw_options)`	Convert options JSON ke dict `{A: "...", B: "..."}`
`snapshot_question_content_hash(question)`	SHA-256 konten soal
`item_content_hash(item)`	SHA-600 konten Item (untuk compare)
`snapshot_slot_map(snapshot)`	Mapping `source_question_id → slot`
`mark_item_variants_stale(item)`	Tandai AI variants stale
`restore_item_variants(item)`	Restore variants dari stale

Normalisasi Field

Beberapa field Sejoli punya variasi nama. Helper fallback:

python
def _question_slot(question: dict, fallback: int) -> int | None:
    """Cari slot dari berbagai kemungkinan key."""
    for key in ("slot", "slot_number", "number", "order", "position", "nomor"):
        raw = question.get(key)
        if raw is not None and raw != "":
            try:
                slot = int(raw)
                return slot if slot > 0 else None
            except (TypeError, ValueError):
                return None
    return fallback   # default: index urutan

def _normalize_options(raw_options):
    """Parse options dari list dict atau list scalar."""
    normalized = []
    for option in raw_options or []:
        if not isinstance(option, dict):
            continue
        increment = _normalize_whitespace(option.get("increment")).upper()  # "A", "B", ...
        text = _normalize_whitespace(option.get("text") or option.get("label"))
        if increment:
            normalized.append({"increment": increment, "text": text})
    return normalized

Edge Cases

Kasus	Penanganan
JSON tidak ada key `tryouts`	Raise `TryoutImportError("Payload must contain non-empty 'tryouts' object")`
Multiple tryouts di 1 JSON, tanpa `target_tryout_id`	Raise error "Expected exactly one tryout"
Soal tanpa `id`	Pakai fallback `"row-{index}"`
Soal tanpa `slot`	Pakai index urutan (1-based)
Option tanpa `increment`	Skip option
Soal duplikat di JSON yang sama	Auto-increment slot (mis. slot 5 ada 2x → jadi 5 dan 6)
Tryout belum ada di App DB	Buat baru otomatis saat commit
Tryout sudah ada, soal dihapus di Sejoli	Tidak auto-delete di App DB — perlu manual cleanup

Endpoint API

Method	Endpoint	Status
`POST`	`/api/v1/import-export/tryout-json/preview`	✅ Implemented
`POST`	`/api/v1/import-export/tryout-json`	✅ Implemented

Detail: lihat API → Admin.

Dependency

hashlib (SHA-256)
json (parse)
sqlalchemy async session
Model: Item, Tryout, TryoutImportSnapshot, TryoutSnapshotQuestion

Bacaan Lanjutan

Modul → Excel Import/Export — alternatif import (nice-to-have)
API → Admin — endpoint import-export
Integrasi → Sejoli Tryout — sumber JSON export
Data → Data Models — tabel snapshot

Last updated Jul 25, 2026

PreviousAlur & Algoritma

NextCTT Scoring