Web1.0のサイトから構造化されたデータを取り出す”Webスクレイピング”という技術があります。
過渡期の技術だと思いますが、当面の応用範囲が広くLWLと組み合わせると面白いです。
rubyには、scrAPIやhpricotが見つかりました。
前回、”テクノラティのランキングデータを取得”のために使ってみましたがhpricotの方が直感的で使い易いと思いました。
ランキングの更新を自動化するためにhpricotをKURO-BOX PROのDebian環境にインストールしましたが、あえなく失敗
とりあえず、バグレポートを送っておきました。
JRubyでトライしてみようかな。
インストールしてみるが失敗
$ sudo gem install hpricot Select which gem to install for your platform (arm-linux) 1. hpricot 0.6 (mswin32) 2. hpricot 0.6 (jruby) 3. hpricot 0.6 (ruby) 4. hpricot 0.5 (ruby) 5. hpricot 0.5 (mswin32) 6. Skip this gem 7. Cancel installation > 3 Building native extensions. This could take a while... ERROR: While executing gem ... (Gem::Installer::ExtensionBuildError) ERROR: Failed to build gem native extension. ruby extconf.rb install hpricot extconf.rb:1:in `require': no such file to load -- mkmf (LoadError) from extconf.rb:1 Gem files will remain installed in /var/lib/gems/1.8/gems/hpricot-0.6 for inspection. Results logged to /var/lib/gems/1.8/gems/hpricot-0.6/ext/hpricot_scan/gem_make.out $
開発用でリトライ
ここに開発バージョンのインストール方法が書いてあったので試してみました。
3と6が選択できましたが、どちらも失敗（何の違いもないみたいですね）
$ sudo gem install hpricot --source http://code.whytheluckystiff.net Bulk updating Gem source index for: http://code.whytheluckystiff.net Select which gem to install for your platform (arm-linux) 1. hpricot 0.6 (mswin32) 2. hpricot 0.6 (jruby) 3. hpricot 0.6 (ruby) 4. hpricot 0.6 (jruby) 5. hpricot 0.6 (mswin32) 6. hpricot 0.6 (ruby) 7. Skip this gem 8. Cancel installation > 3 Building native extensions. This could take a while... ERROR: While executing gem ... (Gem::Installer::ExtensionBuildError) ERROR: Failed to build gem native extension. ruby extconf.rb install hpricot --source http://code.whytheluckystiff.net extconf.rb:1:in `require': no such file to load -- mkmf (LoadError) from extconf.rb:1 Gem files will remain installed in /var/lib/gems/1.8/gems/hpricot-0.6 for inspection. Results logged to /var/lib/gems/1.8/gems/hpricot-0.6/ext/hpricot_scan/gem_make.out $ sudo gem install hpricot --source http://code.whytheluckystiff.net Select which gem to install for your platform (arm-linux) 1. hpricot 0.6 (mswin32) 2. hpricot 0.6 (jruby) 3. hpricot 0.6 (ruby) 4. hpricot 0.6 (jruby) 5. hpricot 0.6 (mswin32) 6. hpricot 0.6 (ruby) 7. Skip this gem 8. Cancel installation > 6 Building native extensions. This could take a while... ERROR: While executing gem ... (Gem::Installer::ExtensionBuildError) ERROR: Failed to build gem native extension. ruby extconf.rb install hpricot --source http://code.whytheluckystiff.net extconf.rb:1:in `require': no such file to load -- mkmf (LoadError) from extconf.rb:1 Gem files will remain installed in /var/lib/gems/1.8/gems/hpricot-0.6 for inspection. Results logged to /var/lib/gems/1.8/gems/hpricot-0.6/ext/hpricot_scan/gem_make.out $