#html #data_pipelines #deep_learning #document_ai #document_image_analysis #document_image_processing #document_parser #document_parsing #docx #donut #information_retrieval #langchain #machine_learning #ml #natural_language_processing #nlp #ocr #pdf #pdf_to_json #pdf_to_text #preprocessing

https://github.com/Unstructured-IO/unstructured

#python #document_ai #document_image_analysis #document_layout_analysis #document_parser #document_understanding #layoutlm #nlp #ocr #publaynet #pubtabnet #pytorch #table_detection #table_recognition #tensorflow

https://github.com/deepdoctection/deepdoctection

#python #beit #beit_3 #bitnet #deepnet #document_ai #foundation_models #kosmos #kosmos_1 #layoutlm #layoutxlm #llm #minilm #mllm #multimodal #nlp #pre_trained_model #textdiffuser #trocr #unilm #xlm_e

Microsoft is developing advanced AI models through large-scale self-supervised pre-training across various tasks, languages, and modalities. These models, such as Foundation Transformers (Magneto) and Kosmos-2.5, are designed to be highly generalizable and capable of handling multiple tasks like language understanding, vision, speech, and multimodal interactions. The benefit to users includes state-of-the-art performance in document AI, speech recognition, machine translation, and more, making these models highly versatile and efficient for a wide range of applications. Additionally, tools like TorchScale and Aggressive Decoding enhance stability, efficiency, and speed in model training and deployment.

https://github.com/microsoft/unilm