Home>

I want to get scraped data into a database.

Problems i am experiencing

I know that the image path is available on the console and I can get it, but the image column in the database is null. Actually, it worked fine before installing gem and carrierwave for image uploading, but it seems to have gone wrong after installation.

ANDshoes.image='https: //item-shopping.c.yimg.jp/i/g/e-minerva_9902010186'

Applicable source code
def self.get_content ()
        thisurl = "http://kakaku.com"
        page_url = "/ search_results /% 8CC% 81E% 83V% 83% 85% 81% 5B% 83Y /"
        for page in 1..2 do
            url = thisurl + page_url
            charset = nil
            html = open (url) do | f |
                charset = f.charset # Get character type
                f.read # read html and pass to variable html
            end
            doc = Nokogiri :: HTML.parse (html.toutf8, nil, 'utf-8')
            for i in 1..30 do
                str_num = i.to_s
                fixed_num = "% 02d"% str_num
                all_name = doc.css ('. item' + fixed_num)
                explanation = all_name.css ('li.searchitem'). inner_text
                name = get_name (all_name)
                image = get_image (all_name)
                size = get_size (all_name, explanation)
                type = get_type (all_name, explanation)
                shoe = Shoe.where (name: name, image: image, size_id: size, type_id: type) .first_or_initialize
                shoe.save
            end
            next_html = doc.css ('. pagenation>ul>li.nextPage a')
            # unless next_html
            page_url = next_html.attribute ('href')
        end
    end
def self.get_image (all_name)
        # Get image
        category_image = all_name.css ('. itemphoto a img')
        image = category_image.attribute ('data-original'). value
        return image
end
What I did

shoe = Shoe.where (name: name, image: image, size_id: size, type_id: type) .first_or_initialize
image: Change image to image: name → No change to null (Problem on the database side)
So
Delete/reset image column → no change
Remove unnecessary things that weren't necessary for carrierwave installation before this work

Supplemental information

Development environment: could9
ruby: 2.3.1p112
rails: 4.2.6

  • Answer # 1

    The reason is not clear, but it has been resolved.
    What I tried:
    Delete image column and add as picture column
    This is the same phenomenon
    Delete picture column and add pic column
    This is reflected in the database.

    Maybe you delete and add directly from phpmyadmin
    In the case of RemovexxxFromZZZ from the console
    There may be a difference.