0

scmongo

MongoDB extensions for Scrapy

Github repo
Main metrics
  • Owner: scrapinghub/scmongo
  • Platform:
  • License::
  • Category::
  • Topic:
  • Like:
    0
      Compare:

Github stars Tracking Chart

=======
scmongo

This project contains some extensions for using MongoDB with the Scrapy
web crawling framework.

Requirements

  • Scrapy 0.14 or above
  • pymongo 2.4 or above

Install

Download and run: python setup.py install

Available extensions

Mongo Cache Storage

Module: scmongo.httpcache

A MongoDB backend for HTTP cache storage. It stores responses using GridFS.

To use it, set the following Scrapy setting in your project::

HTTPCACHE_STORAGE = 'scmongo.httpcache.MongoCacheStorage'

Main metrics

Overview
Name With Ownerscrapinghub/scmongo
Primary LanguagePython
Program languagePython (Language Count: 1)
Platform
License:
所有者活动
Created At2011-04-13 15:33:57
Pushed At2014-10-02 19:47:02
Last Commit At2014-10-02 16:47:01
Release Count0
用户参与
Stargazers Count44
Watchers Count5
Fork Count17
Commits Count24
Has Issues Enabled
Issues Count0
Issue Open Count0
Pull Requests Count3
Pull Requests Open Count1
Pull Requests Close Count0
项目设置
Has Wiki Enabled
Is Archived
Is Fork
Is Locked
Is Mirror
Is Private