python中怎么实现文本单词提取和词频统计功能,针对这个问题,这篇文章详细介绍了相对应的分析和解答,希望可以帮助更多想解决这个问题的小伙伴找到更简单易行的方法。
操作:
strip_html(cls, text) 去除html标签
separate_words(cls, text, min_lenth=3) 文本提取
get_words_frequency(cls, words_list) 获取词频
源码:
class DocProcess(object):
@classmethod
def strip_html(cls, text):
"""
Delete html tags in text.
text is String
"""
new_text = " "
is_html = False
for character in text:
if character == "<":
is_html = True
elif character == ">":
is_html = False
new_text += " "
elif is_html is False:
new_text += character
return new_text
@classmethod
def separate_words(cls, text, min_lenth=3):
"""
Separate text into words in list.
"""
splitter = re.compile("\\W+")
return [s.lower() for s in splitter.split(text) if len(s) > min_lenth]
@classmethod
def get_words_frequency(cls, words_list):
"""
Get frequency of words in words_list.
return a dict.
"""
num_words = {}
for word in words_list:
num_words[word] = num_words.get(word, 0) + 1
return num_words
关于python中怎么实现文本单词提取和词频统计功能问题的解答就分享到这里了,希望以上内容可以对大家有一定的帮助,如果你还有很多疑惑没有解开,可以关注天达云行业资讯频道了解更多相关知识。