I'm a beginner who has never used Kaggle.
I would appreciate it if you could think of the image classification competition as an example.

Various deep learning models have been proposed so far, but is it difficult to aim for higher ranks by using them as they are?
Are you all thinking about your own model? If so, what is the flow and way of thinking when creating a model?
I'm sorry if I heard something out of consideration. Thanks for your answer.

  • Answer # 1

    In image-based competitions, it's common to customize your own or commonly used models for the competition.
    Since it is an image, people who use GBDT etc. do not see it very much (almost?).
    After all there are many CNN systems.

    In the Discussion of Kaggle's competition site, there are always "1st place solution", "2nd place solution" ... and higher-ranking solutions, so you may want to take a look there.

    Example Image competition example

  • Answer # 2

    It depends on the meaning of the word "model", but perhaps the definition given by the questioner is "no". In addition, as requested by the questioner, we are answering within the scope of machine learning in general, while also referring to the image system.

    The main flow of machine learning is as follows (excluding fields such as reinforcement learning). It is not a completely linear flow, and depending on the situation, it may go back to the previous stage and improve.

    Creation of features

    Creating a model

    Evaluation of the model

    Model tuning


    hereThe difference between the top teams is the first "feature creation" part.is. At the stage of model creation,Basically, I almost choose a model called GBDT.. As GBDT, xgboost used to be, and LightGBM is said to have a high adoption rate recently.

    Therefore,I don't want to develop new models..

    If the level of devising the layer of the deep neural network also corresponds to the questioner's "self-made model", I sometimes do it. In addition, since the framework of deep neural networks has a high expressive power of "developing layers", there are various levels of ingenuity. From the level of changing the combination of parameters and stacking, you can customize the layer itself, merge the outputs of multiple layers, and so on. Roughly speaking, CNN, RNN, and LSTM are "levels such as devising layers".It is unclear where the "new model" comes fromis.It's possible that a new "ingenuity" made with Kaggle will be fashionably named and recognized as a new model...

    Postscript: Since CNN is the main image system, the above-mentioned "level of devising the layer of deep neural network" is the mainstream. That's exactly what fkubota answered.

    Probably, developing a so-called "new model" that goes beyond the existing level of ingenuity is (rarely) done by research institutes such as companies and universities, and resources and time like Kaggle are limited. I don't think it will be done much in a place close to practice.

    In answering, we refer to the book "Data Analysis Technology that Wins with Kaggle" and multiple Kaggle-related articles such as Qiita, and describe some of the respondents' interpretations. However, we are not looking deeply into Kaggle itself, so there is no guarantee that it will accurately represent the latest state.


    In Nobuhiro Minaka's "World of Statistical Thinking", "It is important to look at the data before thinking about calculations" "Anyway, I would like to do my best to prevent the output from being excreted by feeding it to an appropriate statistical tool. It claims to be "foolish" (the original text is partially shortened to the extent that the respondent does not impair the meaning of the text)."Model" is just a toolis. I think it is better to break the image that "developing some wonderful model is the point of machine learning".