Comments for chrisspenblog

Comment on Handling PostgreSQL Integrity Errors in Django by Marcob

Marcob — Sun, 10 Jan 2010 17:54:14 +0000

This would have helped and saved me a lot of time

I discovered this solution independently and I tried with every combination of transaction, connection, ecc: this was the only workaround that worked.

Comment on Enabling Implicit Cast From Integer To Boolean in PostgreSQL by Chris

Chris — Wed, 02 Dec 2009 05:47:03 +0000

I’m glad you found it useful.

According to http://doxygen.postgresql.org/pg__cast_8h-source.html the three allowable values for pg_cast.castcontext are “e”, “i”, and “a”, where “a” specifies “coercion in context of assignment”.

Comment on Enabling Implicit Cast From Integer To Boolean in PostgreSQL by GH

GH — Wed, 02 Dec 2009 05:29:56 +0000

Thanks for this post, I found it by googling an error message I got while trying to do exactly this type of implicit cast and your code worked like a charm.

Besides “e” and “i” are there other options for “castcontext”?

Comment on How to Extract a Webpage’s Main Article Content by Gerard Gleeson

Gerard Gleeson — Tue, 01 Dec 2009 11:26:49 +0000

I implemeted a very similar idea in java a while ago, while not even nearly as this it might help people
http://www.redbrick.dcu.ie/~gleesog4/Projects/page.html#extractmainarticle

Comment on Handling PostgreSQL Integrity Errors in Django by Stephen

Stephen — Tue, 06 Oct 2009 22:01:35 +0000

Thanks, this helped me!

Comment on How to Extract a Webpage’s Main Article Content by a

Fri, 02 Oct 2009 21:29:26 +0000

seems to only pull out the comments for some pages. example:
http://www.cbsnews.com/blogs/2009/10/02/politics/politicalhotsheet/entry5359359.shtml

Comment on How to Extract a Webpage’s Main Article Content by Dave

Dave — Fri, 22 May 2009 16:43:47 +0000

Yes it seems strange you would go to all this effort, but not publish the code.

I too would appreciate looking at the code

Comment on How to Extract a Webpage’s Main Article Content by Baishampayan Ghose

Baishampayan Ghose — Tue, 24 Mar 2009 13:52:48 +0000

Nice idea. Any chance of opening up the code so that people can learn? There are similar things for Perl, etc. but nothing for Python. So some code would be appreciated. Thanks.

Comment on How to Extract a Webpage’s Main Article Content by High-quality personal filtering « Sri Spot

High-quality personal filtering « Sri Spot — Wed, 17 Sep 2008 14:24:24 +0000

[...] How to Extract a Webpage’s Main Article Content I had an idea to make a personalized news feed reader. Basically, I’d register a bunch of feeds with the application, and rate a few stories as either “good” or “bad”. The application would then use my ratings and the article text to generate a statistical model, apply that model to future articles, and only recommend those it predicted I would rate as “good”. It sounded like a plausible idea. I decided to start a pet project. [...]

Comment on How to Extract a Webpage’s Main Article Content by Ted Dziuba

Ted Dziuba — Tue, 16 Sep 2008 15:02:18 +0000

Hi Chris,

Nice article. Glad to see that somebody else cares about this problem too. Your approach is very clean. We implemented something that makes several passes over the DOM and snips out nodes driven by a few simple heuristics like “text weight” and such. We found that stuff like that works well for maybe 70% of the web pages out there. To reach the other ~30%, we took a data-driven approach (i.e. using the data we already crawled to figure out what markup is).

Still, we can’t get everything right, but it works Well Enough ™.

Also, the semantic search company Twine has attempted something like this.

Cheers,

Ted
(of Persai, now Pressflip)