Q:

How to integrate my own scraper in Rails app?

I have just created a Rails app with a model app/models/post.rb and have written a scraper scrapers/base_scraper.rb (class BaseScraper) that collect data from the target site to the hash variable data. Now I want to insert values of data into the Post model. How to do it properly in Rails? I have heard smth about Rake but have no idea how to utilize it properly. Help me please!

A:

Assuming that data stores just one post and that each of the key stored in the datahash are valid Post fields (column_name), you can do simply this:
Post.create(data)

If you want to launch the whole process from console, you can create a rake task under lib/tasks directory of your process with the following:
# scraper.rake
namespace :scraper do
  desc "Run scraper"
  task :run => :environment do
    data = BaseScraper.your_collect_data_class_method
    Post.create(data) if data
  end
end

task :default => 'scraper:run'

And then run it from console as a rake task with rake scraper
Of course I also assume that scrapers dir is in your Rails load path.
If not, add it to your application.rbfile.
# application.rb
...
module YourApp
  class Application < Rails::Application
    ...
    config.autoload_paths += Dir["#{config.root}/scrapers/"]
    ...
  end
end

