3fi- ddlZddlZddlZddlmZmZmZmZmZm Z ddl m Z m Z ddl mZmZddlmZddlmZdZd Ze d Zd ed ed efdZ dd ed eded eee fdZ d deee deded dfdZ d!d ed ededeeded df dZd ed eefdZ dd ed eded eee fdZ d ed eefdZ! d!d ed ededeeded df dZ" dd ed eded eee fdZ#d ed eefdZ$ d!d ed ededeeded df dZ%y)"N)Any AsyncIteratorIteratorOptionalSequenceTypeVar) RowMappingtext)ProgrammingErrorSQLAlchemyError)PGEngine) PGVectorStorelangchain_pg_collectionlangchain_pg_embeddingTenginecollection_namereturncKdtd}|jj4d{}|jt |d|id{}|j }|j }dddd{td|d|jS7w7S7%#1d{7swY5xYww)a/ Get the collection uuid for a collection present in PGVector tables. Args: engine (PGEngine): The PG engine corresponding to the Database. collection_name (str): The name of the collection to get the uuid for. Returns: The uuid corresponding to the collection. zSELECT name, uuid FROM z WHERE name = :collection_nameNr parameters Collection, z not found.) COLLECTIONS_TABLE_poolconnectexecuter mappingsfetchone ValueErroruuid)rrqueryconnresult result_map result_fetchs h/var/www/auto_recruiter/arenv/lib/python3.12/site-packages/langchain_postgres/utils/pgvector_migrator.py__aget_collection_uuidr(s&&7%88V WE||##%--|| K%6$H$  __& !**, -- <'8 DEE   - ----sV)B>B#B>"B)B%$B)6 B>B'"B>%B)'B>)B;/B2 0B;7B> batch_sizecPK tjt||}dtd}|jj 4d{}|d{}|j t|d|id{} |j|}|sn|D cgc]} | jc} 3dddd{y7y7q7Mcc} w7#1d{7swYyxYw#t$rtd|dt$r} td |d | di| | d} ~ wwxYww)  Extract all data belonging to a PGVector collection. Args: engine (PGEngine): The PG engine corresponding to the Database. collection_name (str): The name of the collection to get the data for. batch_size (int): The batch size for collection extraction. Default: 1000. Optional. Yields: The data present in the collection. zSELECT * FROM z WHERE collection_id = :idNidr)sizer does not exist.z(Failed to extract data from collection 'z': ) statementparamsorig) asyncio create_taskr(EMBEDDINGS_TABLErrrr fetchmany_mappingr r r ) rrr) uuid_taskr"r#r! result_proxyrowsrowes r'__aextract_pgvector_collectionr<+s;"''(>v(WX  !1 22LM<<'') 5 5T"?D!%d5ktTl!SSL#--:->/34s||44  5 5 5"S 5 5 5 5 5 K<'88HIJJ @@QQTUVTWX$<   sD&AC' C C'CC%C;C <CC -C4 C'?CC'D&C'C C CC'C$C C$ C'#D&$C'' D#DD##D& data_batches vector_storemax_concurrencyc~Kt}|23d{}|jtj|j |Dcgc]}|j c}|Dcgc]!}t j|j#c}|Dcgc]}|jc}|Dcgc]}|jc}t||k\stj|tjd{\}}7cc}wcc}wcc}wcc}w76|rtj|d{7yyw)N)texts embeddings metadatasids) return_when)setaddr2 ensure_futureaadd_embeddingsdocumentjsonloads embedding cmetadatar,lenwaitFIRST_COMPLETED)r=r>r?pending batch_datadata_s r'__concurrent_batch_insertrVQs G(j  ! !,,5?@T4==@GQRt 4>> :R:DE$t~~E-78T8 -   wrWrXdestination_tabler7r"r#r!r$r%collection_data_lenr= table_sizes r'__amigrate_pgvector_collectionrejsR(%335##$:6?$STI  011TU ||##%44||DK_d $K/ L7:K;?L7:K;'L7"K#L7&K&K $K&) L74K#5A%L7K<L7"LK?LLL L7(L),L7LL7"L";L<L"LL" L7#L $L7;K>KL7K K K L7L7 K&#L7&K9,K/ -K94 L7?LLL7L L LL7L"L" L7"L4(L+ )L40L7cK dt}|jj4d{}|jt |d{}|j }|j }dddd{Dcgc]}|d c}S7i7I7#1d{7swY+xYwcc}w#t$r}tdt|z|d}~wwxYww)5Lists all collection names present in PGVector table.zSELECT name from Nnamez2Please provide the correct collection table name: ) rrrrr rfetchallr r str)rr"r#r$r%all_rowsr:r;s r'!__alist_pgvector_collection_namesrls #$5#67<<'') - -T<<U 44F*J!**,H - -(00F 00  -4 - - - -1  @3q6 I  sC'B5BB5B B$B2 B5=B>B5 B0B5CB5BB5B-!B$ "B-) B55 C>CCCcKt|||} |j|jd{}|.7 #t$rYywxYwwr+N)r< _run_as_async __anext__StopAsyncIterationrrr)iteratorr$s r'aextract_pgvector_collectionrtsX".fozRH  !//0B0B0DEEFL E!   s5A"A? AAA A  A A  AcRK|jt|d{S7w)rgN)rorlrs r'alist_pgvector_collection_namesrws%%%&G&OP PP Ps '%'c \K|jt|||||d{y7w)a Migrate all data present in a PGVector collection to use separate tables for each collection. The new data format is compatible with the PGVectorStore interface. Args: engine (PGEngine): The PG engine corresponding to the Database. collection_name (str): The collection to migrate. vector_store (PGVectorStore): The PGVectorStore object corresponding to the new collection table. use_json_metadata (bool): An option to keep the PGVector metadata as json in the new table. Default: False. Optional. delete_pg_collection (bool): An option to delete the original data upon migration. Default: False. Optional. insert_batch_size (int): Number of rows to insert at once in the table. Default: 1000. N)rorerrr>rWrXs r'amigrate_pgvector_collectionrzs7,   &       s ",*,c#Kt|||} |j|j}|%#t$rYywxYwwrn)r< _run_as_syncrprqrrs r'extract_pgvector_collectionr}sT".fozRH  ((););)=>FL "   s$A#6A AAAAc6|jt|S)rg)r|rlrvs r'list_pgvector_collection_namesr s   @H IIc @|jt|||||y)a Migrate all data present in a PGVector collection to use separate tables for each collection. The new data format is compatible with the PGVectorStore interface. Args: engine (PGEngine): The PG engine corresponding to the Database. collection_name (str): The collection to migrate. vector_store (PGVectorStore): The PGVectorStore object corresponding to the new collection table. delete_pg_collection (bool): An option to delete the original data upon migration. Default: False. Optional. insert_batch_size (int): Number of rows to insert at once in the table. Default: 1000. N)r|rerys r'migrate_pgvector_collectionr%s)( &      r))r[)Fr)&r2rKr^typingrrrrrr sqlalchemyr r sqlalchemy.excr r v2.enginerv2.vectorstoresrrr4rrjr(intr<rVboolrelistrlrtrwrzr}rrrr'rso LL'< +-+ CL  8# ###8J'( #R$ 45$$$ $:,1! FN FNFN FN#4. FN  FN  FNR  #Y( 8J'( 4Q Q #YQ,1!   #4.     H hz"# 4J8JS J,1!   #4.     r