数据集 | 6.6亿条美国谷歌地图评论数据(~2021.9)

数据集名: 美国谷歌地图评论数据集
数据来源: Google Map
覆盖日期: 2008-01-25 ~ 2021-09 
评论数量: 666,324,103(6.6亿)
用户数量: 113,643,107(1.1亿用户)
实体数量: 4,963,111家商铺
所含字段: 评分、文本、图片、地址、地理信息、价格、营业时间等
数据格式: json文件
下载数据: 
    - https://datarepo.eng.ucsd.edu/mcauley_group/gdrive/googlelocal/
    - https://mcauleylab.ucsd.edu/public_datasets/gdrive/googlelocal/
本文声明: 科研用途； 如分享有问题，可加微信372335839，备注「姓名-学校-专业」

[1]Li, Jiacheng, Jingbo Shang, and Julian McAuley. "UCTopic: Unsupervised contrastive learning for phrase representations and topic mining." arXiv preprint arXiv:2202.13469 (2022).
[2]Yan, An, Zhankui He, Jiacheng Li, Tianyang Zhang, and Julian McAuley. "Personalized showcases: Generating multi-modal explanations for recommendations." In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieval, pp. 2251-2255. 2023.

# 读取meta-New_York.json.gz 前1行记录
first_n_json_objects = read_json_nlines(file='meta-New_York.json.gz', nrows=1)
first_n_json_objects

[{'name': 'A-Top Insurance',
  'address': 'A-Top Insurance, 1009 Brighton Beach Ave, Brooklyn, NY 11235',
  'gmap_id': '0x89c24469c758686b:0x641f5b84cb9bedfa',
  'description': None,
  'latitude': 40.5782537,
  'longitude': -73.9591269,
  'category': ['Insurance broker', 'Insurance agency'],
  'avg_rating': 2,
  'num_of_reviews': 4,
  'price': None,
  'hours': [['Thursday', '10AM–6PM'],
   ['Friday', '10AM–6PM'],
   ['Saturday', 'Closed'],
   ['Sunday', 'Closed'],
   ['Monday', '10AM–6PM'],
   ['Tuesday', '10AM–6PM'],
   ['Wednesday', '10AM–6PM']],
  'MISC': None,
  'state': 'Open ⋅ Closes 6PM',
  'relative_results': ['0x89c24449907718fb:0x31b554a0983f621d',
   '0x4065f38ac8af66fd:0x991c223e83658501',
   '0x89c24450d471cd67:0x169d824916e1d42',
   '0x89c25bae39b5a677:0x5f129fa988693a25',
   '0x89c245029897765d:0xa04072e1ef9ab823'],
  'url': 'https://www.google.com/maps/place//data=!4m2!3m1!1s0x89c24469c758686b:0x641f5b84cb9bedfa?authuser=-1&hl=en&gl=us'}]

%%time
import pandas as pd

#读取前2行 
#mdf = pd.read_json('meta-New_York.json.gz', nrows=2, compression='gzip', lines=True)

#全部读取
mdf = pd.read_json('meta-New_York.json.gz', compression='gzip', lines=True)
mdf

CPU times: user 2.65 s, sys: 2.71 s, total: 5.36 s
Wall time: 2.55 s

name             商业实体名称
address          地址
gmap_id          商业实体ID
description      业务描述
latitude         纬度
longitude        经度
category         商业的类别
avg_rating       该业务的平均评分
num_of_reviews   评论数量
price            业务价格
hours            营业时间
MISC             MISC信息
state            当前状态（例如，永久关闭）
relative_results 谷歌推荐的相关商业信息
url              企业的网址

import missingno as ms

ms.matrix(mdf)

import gzip
import jsonlines


def read_json_nlines(file, nrows):
    json_objects = []
    with gzip.open(file, 'rt', encoding='utf-8') as gz_file:
        reader = jsonlines.Reader(gz_file)
        for i in range(nrows):
            try:
                json_objects.append(reader.read())
            except EOFError:
                break # 如果到达文件末尾，则退出循环
    return json_objects

# 读取review-New_York.json.gz 前2行记录
first_n_json_objects = read_json_nlines(file='review-New_York.json.gz', nrows=2)
first_n_json_objects

[
  {'user_id': '101855823232666695168',
  'name': 'R Mac',
  'time': 1629141186463,
  'rating': 1,
  'text': 'Natalia may be the worst agent I have ever dealt with. Look up the definition of entitled b**** in the dictionary and there she would be. Look at their reviews...they are garbage through and through.',
  'pics': None,
  'resp': None,
  'gmap_id': '0x89c24469c758686b:0x641f5b84cb9bedfa'},

 {'user_id': '105821946869087882225',
  'name': 'Beck TJ',
  'time': 1528477593994,
  'rating': 1,
  'text': 'The lady at the front desk is rude. The bathroom key is supposed to be available for anyone who is visiting the business and any question must be answered in a nice manner.',
  'pics': None,
  'resp': None,
  'gmap_id': '0x89c24469c758686b:0x641f5b84cb9bedfa'}
  
  ]

%%time
import pandas as pd

#读取前2行 
#rdf = pd.read_json('review-New_York.json.gz', nrows=2, compression='gzip', lines=True)
#全部读取
rdf = pd.read_json('review-New_York.json.gz', compression='gzip', lines=True)
rdf

CPU times: user 1min 30s, sys: 2min 2s, total: 3min 32s
Wall time: 4min 11s

user_id   评论者的 ID
name      评论者的姓名
time      评论时间（unix 时间戳）
rating    对企业的评分
text      评论文本
pics      评论图片链接
resp      商家对评论的回复，包括 unix 时间戳和回复文本
gmap_id   商业 ID

import missingno as ms

ms.matrix(rdf)