skipping?

[demonaxe]

how do you change it to not skip other weblinks?

[kezakez]

it skips processed links and anything that isn't http.
https://github.com/kezakez/python-web-crawler/blob/master/crawl.py#L9