Json To Pydantic

#Overview

The JsonToPydantic class was developed to make it easier to dynamically convert JSON dictionaries into Pydantic models with automatic field typing. This process is very useful when working with unstructured or variable JSON data, and there is a need to validate and manipulate that data using the robustness that Pydantic offers.

Basically, the class lets you turn an arbitrary JSON into a Pydantic class generated at runtime, inferring the basic types of the values (such as string, integer, float, boolean, lists and dictionaries). This simplifies workflows that involve data validation, testing or manipulation where the JSON schema is not defined beforehand.

In practice, just create an instance of JsonToPydantic, pass the JSON as a dictionary to the class methods, and automatically get typed models and instances, ready for validation and easier access to the data.

#Execution Flow

1. Instantiates the JsonToPydantic class, optionally passing a name for the Pydantic model that will be created (model_name).
2. Calls the build_model method passing a JSON dictionary to create a dynamic Pydantic model. This method infers the field types based on the values in the dictionary.
3. Uses the parse method to create an instance of the previously generated Pydantic model, filled with the data from the JSON dictionary.
4. If any step fails (type inference, model construction, instantiation), the class raises RuntimeError exceptions to indicate explanatory errors.

#Class Methods Table

#Key Architecture Points and Insights

• The class uses the pydantic library to create models dynamically (create_model) and for robust data validation.
• The inference done by the _infer_type method covers common Python types and returns Any when the type is not clearly identified.
• The use of RuntimeError exceptions with specific messages makes it easier to understand possible failures in the flow.
• The class does not depend on external environment variables to work.
• The approach adopted makes it easier to create dynamic schemas, which is useful for systems that consume JSONs with a variable structure or one unknown at development time.

#Class and Methods Description

#JsonToPydantic Class

#Description

Class for dynamically creating Pydantic models from JSON dictionaries, allowing automatic validation and typing of the fields according to the data received.

#Constructor Arguments

#Methods

#1. __init__

Description

Initializes the object by setting the name of the Pydantic model that will be created dynamically.

Arguments

• model_name (str): Name of the Pydantic model (optional).

Returns

• Returns no value.

Raises

• None.

Examples

converter = JsonToPydantic()  # modelo padrão "DynamicModel"
converter_custom = JsonToPydantic("MeuModelo")

#2. _infer_type

Description

Identifies the Python type corresponding to the given value to define the type of the field in the model.

Arguments

• value (Any): Value to be analyzed and typed.

Returns

• Type: Corresponding Python type (str, int, float, bool, list, dict or Any).

Raises

• RuntimeError: If an error occurs during type inference.

Examples

tipo = converter._infer_type("texto")  # retorna <class 'str'>
tipo = converter._infer_type(123)      # retorna <class 'int'>

#3. build_model

Description

Builds and returns a dynamically generated Pydantic model with fields and types based on the given JSON dictionary.

Arguments

• data (Dict[str, Any]): Dictionary with data for inferring the model.

Returns

• Type[BaseModel]: Class of the Pydantic model created.

Raises

• RuntimeError: If an error occurs during model creation.

Examples

modelo = converter.build_model({"nome": "Ana", "idade": 30})
# Retorna um modelo equivalente a:
# class DynamicModel(BaseModel):
#     nome: str
#     idade: int

#4. parse

Description

Generates a Pydantic model from the data and returns an instance filled with that data.

Arguments

• data (Dict[str, Any]): JSON dictionary to be converted into a model instance.

Returns

• BaseModel: Instance of the Pydantic model with the validated data.

Raises

• RuntimeError: If a failure occurs in creating or validating the instance.

Examples

if __name__ == "__main__":
    data = {
        "text": "A empresa TechNova está crescendo rapidamente.",
        "task": "Se o nome da empresa for TechNova, troque por BetterAI"
    }
    parser = JsonToPydantic("ResearchRequest")
    request = parser.parse(data)

    print(request)
    print(type(request))

# python -m src.content_parse.pydantic_shema

#Mapping of schema forms

#1. Inputs accepted by the route

Request format: multipart/form-data.

Fields:

• job_id (required): str
• metadata (required): str containing valid JSON
• document_schema (required): str containing valid JSON
• file (required): file
• config (optional): str containing valid JSON

File rules on this route:

• Allowed extensions: txt, md, pdf, docx
• Maximum size: 50 MB

Common input errors:

• Invalid JSON in metadata
• Invalid JSON in schema
• Invalid JSON in config

#2. Important: who interprets the document_schema

• document_schema is not converted by JsonToPydantic.
• On the route, it is converted by GeneratePydanticSchema + FieldMetadataParser.
• JsonToPydantic is used in the agent for input_data and config_data.

#3. All forms of document_schema accepted in practice

#3.1 Simple declarative field

{
    "summary": {
        "type": "str",
        "description": "Resumo do conteúdo do arquivo"
    }
}

#3.2 Declarative field with required, default, example

{
    "title": {
        "type": "str",
        "required": true,
        "description": "Título principal",
        "example": "Relatório de Q2"
    },
    "confidence": {
        "type": "float",
        "required": false,
        "default": 0.0,
        "description": "Confianca da extração"
    }
}

#3.3 Declarative field with size validation

{
    "abstract": {
        "type": "str",
        "description": "Resumo detalhado",
        "min_length": 20,
        "max_length": 500
    }
}

#3.4 Declarative list of primitives

{
    "keywords": {
        "type": "list",
        "items": {
            "type": "str"
        },
        "description": "Palavras-chave"
    }
}

#3.5 Declarative list of objects

{
    "entities": {
        "type": "list",
        "items": {
            "type": "object",
            "properties": {
                "name": {
                    "type": "str",
                    "description": "Nome"
                },
                "category": {
                    "type": "str",
                    "description": "Categoria"
                },
                "score": {
                    "type": "float",
                    "description": "Pontuação"
                }
            }
        }
    }
}

#3.6 Automatic inference by example (without type)

{
    "summary": "texto exemplo",
    "score": 0.98,
    "approved": true
}

#3.7 Nested object inference

{
    "invoice": {
        "number": "INV-001",
        "total": 199.9,
        "paid": false
    }
}

#3.8 List of objects inference

{
    "items": [
        {
            "sku": "A-1",
            "quantity": 2,
            "unit_price": 49.9
        }
    ]
}

#3.9 Inference with an empty list

{
    "items": []
}

In this case, the type becomes a generic list.

#3.10 Hybrid schema (declarative + inference in the same payload)

{
    "summary": {
        "type": "str",
        "description": "Resumo final"
    },
    "stats": {
        "pages": 12,
        "language": "pt-BR"
    }
}

#4. Type mapping in declarative mode

Values recognized in type:

• str
• int
• float
• bool
• list
• dict

If the type is not recognized, it falls back to Any.

#5. Important rules and limitations

1. If the field is a dict and has a type key, it is treated as declarative metadata.
2. For type = "list", items is required.
3. A declarative list of objects requires items.type = "object" with properties.
4. Fields without required are optional by default.
5. The parser accepts a hybrid structure, mixing declarative and inferred fields.