Python website crawler.
This is an open source, multi-threaded website crawler written in Python. There is still a lot of work to do, so feel free to help out with development.
Note: This is part of an open source search engine. The purpose of this tool is to gather links only. The analytics, data harvesting, and search algorithms are being created as separate programs.
| Name With Owner | buckyroberts/Spider |
|---|---|
| Primary Language | Python |
| Program language | Python (Language Count: 1) |
| Platform | |
| License: |
| Created At | 2016-02-24 21:44:33 |
|---|---|
| Pushed At | 2023-01-02 10:54:11 |
| Last Commit At | 2016-05-04 17:27:14 |
| Release Count | 0 |
| Stargazers Count | 1k |
|---|---|
| Watchers Count | 110 |
| Fork Count | 659 |
| Commits Count | 68 |
| Has Issues Enabled | |
| Issues Count | 43 |
| Issue Open Count | 30 |
| Pull Requests Count | 13 |
| Pull Requests Open Count | 53 |
| Pull Requests Close Count | 26 |
| Has Wiki Enabled | |
|---|---|
| Is Archived | |
| Is Fork | |
| Is Locked | |
| Is Mirror | |
| Is Private |