Comments on: What I learned by information retrieval in one week http://zeta-puppis.com/2008/10/19/what-i-learned-by-information-retrieval-in-one-week/ my very own personal corner Thu, 29 Jul 2010 09:55:57 +0000 http://wordpress.org/?v=2.9.2 hourly 1 By: Zeta-Puppis.com » Optimize your programs http://zeta-puppis.com/2008/10/19/what-i-learned-by-information-retrieval-in-one-week/comment-page-1/#comment-732 Zeta-Puppis.com » Optimize your programs Tue, 02 Dec 2008 20:17:49 +0000 http://zeta-puppis.com/?p=159#comment-732 [...] The last time I blogged about a new course I’m fol­low­ing at my uni­ver­sity. This course, held by Pasquale Lops and Gio­vanni Semer­aro, is very inter­est­ing at the point that I’ll be devel­op­ing a custom infor­ma­tion retrieval engine as part of my intern­ship project. I can’t tell much more at this point since the intern­ship haven’t started yet and I’m not sure I can release more details about this project (we’re still in the process of decid­ing if and how the whole thing will be released to the world). [...] […] The last time I blogged about a new course I’m fol­low­ing at my uni­ver­sity. This course, held by Pasquale Lops and Gio­vanni Semer­aro, is very inter­est­ing at the point that I’ll be devel­op­ing a custom infor­ma­tion retrieval engine as part of my intern­ship project. I can’t tell much more at this point since the intern­ship haven’t started yet and I’m not sure I can release more details about this project (we’re still in the process of decid­ing if and how the whole thing will be released to the world). […]

]]> By: Giuliani Vito, Ivan http://zeta-puppis.com/2008/10/19/what-i-learned-by-information-retrieval-in-one-week/comment-page-1/#comment-584 Giuliani Vito, Ivan Mon, 20 Oct 2008 17:04:54 +0000 http://zeta-puppis.com/?p=159#comment-584 I just made research.google.com my homepage :) Anyway thanks for the good advices, I'm looking forward to this stuff right now. Thanks again! I just made research.google.com my homepage :)
Anyway thanks for the good advices, I’m looking forward to this stuff right now. Thanks again!

]]> By: CJ http://zeta-puppis.com/2008/10/19/what-i-learned-by-information-retrieval-in-one-week/comment-page-1/#comment-581 CJ Mon, 20 Oct 2008 09:24:55 +0000 http://zeta-puppis.com/?p=159#comment-581 P.S: you'll learn an awful lot by reading Google patents and looking at how their algo works (within the limitations of the knowledge shared by them) -look at SIGIR papers too, and go if you can, it's a great conference. cj P.S: you’ll learn an awful lot by reading Google patents and looking at how their algo works (within the limitations of the knowledge shared by them) -look at SIGIR papers too, and go if you can, it’s a great conference.

cj

]]> By: CJ http://zeta-puppis.com/2008/10/19/what-i-learned-by-information-retrieval-in-one-week/comment-page-1/#comment-580 CJ Mon, 20 Oct 2008 09:22:34 +0000 http://zeta-puppis.com/?p=159#comment-580 Hi, I'm finishing a PhD in natural language processing (natural language generation and understanding). I spent my first few years building search engines, and studied them in great depth. You'll find that the guys who work in SEO (search engine optimisation) have a pretty good understanding of how the Google algorithm works, PageRank, stemming, tagging, and all that stuff. I also wrote a stemmer in my first year which is pretty simple really but has been really useful for me, as it stems to actual words, in their proper natural form. Quite important for NLG. http://fizz.cmp.uea.ac.uk/Research/stemmer/ You'll find that tf-idf has some limitations, such as not working very well on large document collections, it's also not very good with synonyms so it's hard for this method to find relationships between words. N-grams - look at Witten-Bell discounting and smoothing. Check out LSA too, although it also has some limitations but fewer. You can use neural-networks. On my blog you'll find some links to good NLP/AI tools: http://scienceforseo.blogspot.com/2008/10/top-freeware-stuff.html Good work, and enjoy your journey in NLP/IR! Hi,

I’m finishing a PhD in natural language processing (natural language generation and understanding). I spent my first few years building search engines, and studied them in great depth. You’ll find that the guys who work in SEO (search engine optimisation) have a pretty good understanding of how the Google algorithm works, PageRank, stemming, tagging, and all that stuff.

I also wrote a stemmer in my first year which is pretty simple really but has been really useful for me, as it stems to actual words, in their proper natural form. Quite important for NLG.

http://fizz.cmp.uea.ac.uk/Research/stemmer/

You’ll find that tf-idf has some limitations, such as not working very well on large document collections, it’s also not very good with synonyms so it’s hard for this method to find relationships between words.

N-grams - look at Witten-Bell discounting and smoothing.

Check out LSA too, although it also has some limitations but fewer. You can use neural-networks.

On my blog you’ll find some links to good NLP/AI tools:

http://scienceforseo.blogspot.com/2008/10/top-freeware-stuff.html

Good work, and enjoy your journey in NLP/IR!

]]>