3fi0 dZddlmZmZmZmZmZmZddlm Z m Z ddl m Z ddl mZddlmZGdde eZ dd d d eeeefeefd e d eedeedef dZy )z6Document transformers that use OpenAI Functions models)AnyDictOptionalSequenceTypeUnion)BaseDocumentTransformerDocument)BaseLanguageModel)ChatPromptTemplate) BaseModelc^eZdZUdZeed< deededeefdZdeededeefdZ y) OpenAIMetadataTaggeraExtract metadata tags from document contents using OpenAI functions. Example: .. code-block:: python from langchain_community.chat_models import ChatOpenAI from langchain_community.document_transformers import OpenAIMetadataTagger from langchain_core.documents import Document schema = { "properties": { "movie_title": { "type": "string" }, "critic": { "type": "string" }, "tone": { "type": "string", "enum": ["positive", "negative"] }, "rating": { "type": "integer", "description": "The number of stars the critic rated the movie" } }, "required": ["movie_title", "critic", "tone"] } # Must be an OpenAI model that supports functions llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613") tagging_chain = create_tagging_chain(schema, llm) document_transformer = OpenAIMetadataTagger(tagging_chain=tagging_chain) original_documents = [ Document(page_content="Review of The Bee Movie By Roger Ebert This is the greatest movie ever made. 4 out of 5 stars."), Document(page_content="Review of The Godfather By Anonymous This movie was super boring. 1 out of 5 stars.", metadata={"reliable": False}), ] enhanced_documents = document_transformer.transform_documents(original_documents) tagging_chain documentskwargsreturnc g}|D]]}|jj|j}t|ji||j}|j |_|S)zgAutomatically extract and populate metadata for each document according to the provided schema.) page_contentmetadata)rrunrr rappend)selfrr new_documentsdocumentextracted_metadata new_documents x/var/www/auto_recruiter/arenv/lib/python3.12/site-packages/langchain_community/document_transformers/openai_functions.pytransform_documentsz(OpenAIMetadataTagger.transform_documents4st  ! /H'+'9'9'='=h>S>S'T #%22D.D(2C2CDL   .  /cKtwN)NotImplementedError)rrrs ratransform_documentsz)OpenAIMetadataTagger.atransform_documentsEs"!s N) __name__ __module__ __qualname____doc__r__annotations__rr rr$r rrr sa#J@!(+7: ( ""!(+"7:" ( "r rN)tagging_chain_kwargsmetadata_schemallmpromptr+rcddlm}t|tr|n|j }|xsi}|||fd|i|}t |S)aN Create a DocumentTransformer that uses an OpenAI function chain to automatically tag documents with metadata based on their content and an input schema. Args: metadata_schema: Either a dictionary or pydantic.BaseModel class. If a dictionary is passed in, it's assumed to already be a valid JsonSchema. For best results, pydantic.BaseModels should have docstrings describing what the schema represents and descriptions for the parameters. llm: Language model to use, assumed to support the OpenAI function-calling API. Defaults to use "gpt-3.5-turbo-0613" prompt: BasePromptTemplate to pass to the model. Returns: An LLMChain that will pass the given function to the model. Example: .. code-block:: python from langchain_community.chat_models import ChatOpenAI from langchain_community.document_transformers import create_metadata_tagger from langchain_core.documents import Document schema = { "properties": { "movie_title": { "type": "string" }, "critic": { "type": "string" }, "tone": { "type": "string", "enum": ["positive", "negative"] }, "rating": { "type": "integer", "description": "The number of stars the critic rated the movie" } }, "required": ["movie_title", "critic", "tone"] } # Must be an OpenAI model that supports functions llm = ChatOpenAI(temperature=0, model="gpt-3.5-turbo-0613") document_transformer = create_metadata_tagger(schema, llm) original_documents = [ Document(page_content="Review of The Bee Movie By Roger Ebert This is the greatest movie ever made. 4 out of 5 stars."), Document(page_content="Review of The Godfather By Anonymous This movie was super boring. 1 out of 5 stars.", metadata={"reliable": False}), ] enhanced_documents = document_transformer.transform_documents(original_documents) r)create_tagging_chainr.)r))langchain_classic.chains.openai_functionsr0 isinstancedictschemar)r,r-r.r+r0_tagging_chain_kwargsrs rcreate_metadata_taggerr6KsdpO ot ,   # # % 16B(%+/DM m <r=s<==F<5="2I="F,0C= ,0 C=4S>4 ?:;C= C= ' (C= #4. C=  C=r