python3.6爬虫需要安装的模块

  • 时间:
  • 浏览:2
  • 来源:彩神大发欢乐生肖_神彩大发欢乐生肖官方

        phantomjs(无界面浏览器)

    D:\Program Files\MongoDB\Server\3.6\bin>mongod --bind_ip 0.0.0.0 --logpath C:\data\logs\logs.txt  --logappend --dbpath C:\data\db --port 27017 --serviceName "mongodb" --serviceDisplayName "mongodb" --install

            用下面的土最好的法律法律依据安装

        selenium

    D:\Program Files\MongoDB\Server\3.6\bin>"D:\Program Files\MongoDB\Server\3.6\bin\mongod.exe" --dbpath ../data/db

        pymongo

redis数据库安装:

    不需要 安装的库

    建立日志文件在c盘创建C:\data\db文件夹和C:\data\logs\logs.txt文件

            pip3 install pyquery

            http://npm.taobao.org/mirrors/chromedriver/

                链接:https://pan.baidu.com/s/1oAh2Dse 密码:okk0

        chromedriver

                http://www.lfd.uci.edu/~gohlke/pythonlibs/#lxml

        re

            pip3 install beautifulsoup4

            首先安装

                https://pypi.python.org/pypi/pyOpenSSL#downloads

    框架:

                链接:https://pan.baidu.com/s/1mj2VjxI 密码:ls54

            pip3 install pymysql

    存储库:

MongoDB 的安装:

            pip3 install pymongo

    https://download.robomongo.org/1.2.1/windows/robo3t-1.2.1-windows-x86_64-3e50a65.exe

        django

  内置库

            2. lxml

        lxml

                pip install wheel

            pip3 install PySpider

            6. Scrapy

            有之前 下载失败

            pip3 install requests

            pip3 install django

                http://www.lfd.uci.edu/~gohlke/pythonlibs/#twisted

                pip3 install scrapy

启动服务:

            https://pypi.python.org/pypi/lxml

        requests

            5. Pywin32

    http://blog.51cto.com/tdcqvip/2072845

        pymysql

            pip3 install lxml

        jupyter(强大的记事本)

            http://phantomjs.org/

        beautifulsoup4(依赖lxml)

                https://sourceforge.net/projects/pywin32/files/pywin32/Build%20221/pywin32-221.win-amd64-py3.6.exe/download

            pip3 install  selenium

        PySpider

配置可视化服务:

            jupyter notebook(启动服务)

            下载驱动,放入配置好的环境变量下

本文转自 天道酬勤VIP 51CTO博客,原文链接:http://blog.51cto.com/tdcqvip/2073149

            4. Twisted

        urllib

            pip3 install jupyter

        flask(代理IP的库)

                https://sourceforge.net/projects/pywin32/files/pywin32/Build%20220/

        scrapy

                软件和再次再次出现问提修复脚本,

图形化管理页面软件

            pip3 install wheel

        pyquery

            3. PyOpenssl

        redis

参考上篇博客:    

            有之前 载pip3 install 下载的文件路径和名字

            下载,有之前 配置环境变量

            1. wheel

            pip3 install flask

            pip3 install redis