文本分析

如何从网络世界中高效地采集数据？是否能从文本中挖掘出人类的偏见等认知信息？如何从杂乱的文本数据中抽取文本信息(变量)？本文汇总的列表将让你对文本、对Python文本分析个全面的了解...

如何使用Python从网络中爬取数据，如何从文本数据中抽取信息。本文汇总了常见的python代码案例，方便大家快速学习...

非结构文本、图片、视频等数据是待挖掘的数据矿藏， 在经管、社科等研究领域中谁拥有了从非结构提取结构化信息的能力，谁就拥有科研上的数据优势。正则表达式是一种强大的文档解析工具，但它们常常难以应对现实世界文档的复杂性和多变性。而随着chatGPT这类LLM的出现，为我们提供了更强大、更灵活的方法来处理多种类型的文档结构和内容类型。For many years, regular expressions have been my go-to tool for parsing documents, and I am sure it has been the same for many other technical folks and industries.Even though regular expressions are powerful and successful in some case, they often struggle with the complexity and variability of real-world documents.Large language models on the other end provide a more powerful, and flexible approach to handle many types of document structures and content types....

开源 LLMS 越来越受欢迎，Ollama 的 OpenAI 兼容性后来发布了，这使得使用 JSON 模式获取结构化输出成为可能。在本篇博文的结尾，您将了解如何有效地利用 Instructor 和 ollama。但在继续之前，让我们先探讨一下修补的概念。Open-source LLMS are gaining popularity, and the release of Ollama's OpenAI compatibility later it has made it possible to obtain structured outputs using JSON schema.By the end of this blog post, you will learn how to effectively utilize instructor with ollama. But before we proceed, let's first explore the concept of patching....

情感分析是分析文本以确定消息的情绪基调是积极、消极还是中性的过程。通过情感分析，我们可以了解文本是否表现出快乐、悲伤、愤怒等情绪。主要的计算方法有语义词典法、机器学习法、混合方法、其他方法。 随着chatGPT这类大语言模型的出现， 它们增强了文本理解能力，使我们能够更精准的把握文本中的语义和情绪，也因此大型语言模型 (LLM) 一出场就有实现情感分析功能。Sentiment analysis is the process of analyzing text to determine whether the emotional tone of a message is positive, negative, or neutral. Through sentiment analysis, we can understand whether the text expresses emotions such as happiness, sadness, anger, etc. The main computational methods are semantic dictionary method, machine learning method, hybrid method, and other methods. With the emergence of large language models such as chatGPT, they enhance text understanding capabilities, allowing us to more accurately grasp the semantics and emotions in the text. Therefore, large language models (LLMs) have implemented sentiment analysis functions as soon as they appeared....