Home>

Currently, I am trying to estimate the position using only images.
I think that it will be easier to estimate by using two images (before and after myself) for input.
I'm doing it, but I get the following error and I don't know what to do.
I would like you to tell me.

If there are any other strange points, I would appreciate it if you could let me know.

* The classes of classes are changed appropriately.

ValueError: Layer reshape_1 was called with an input that isn't a symbolic tensor. Received type:<class'builtin_function_or_method'>. Full input: [<built-in function input>]. All inputs to the layer should be tensors.
Corresponding source code
from keras import backend as K
from keras.datasets import mnist
from keras.layers import Activation, Add, BatchNormalization, Dense, Dropout, Input, Concatenate, Flatten, Reshape
from keras.models import Model
from keras.utils.np_utils import to_categorical
import matplotlib.pyplot as plt
import numpy as np
import glob
from PIL import Image
from keras.utils import np_utils
from sklearn.model_selection import train_test_split
from keras.preprocessing.image import load_img, img_to_array
classes = [
           "Dog", "Cat", "Rabbit"
           ]
X = []
Y = []
for index, classlabel in enumerate (classes):
    dir = "./" + classlabel
    files = glob.glob (dir + "/*.jpg")
    for i, file in enumerate (files):
        image = Image.open (file)
        image = image.convert ("RGB")
        image = image.resize ((64, 64))
        data = np.asarray (image)
        X.append (data)
        Y.append (index)
X = np.array (X)
Y = np.array (Y)
X = X.astype ('float32') /255.0
Y = np_utils.to_categorical (Y, len (classes))
x_train, x_test, y_train, y_test = train_test_split (X, Y, test_size = 0.10)

    # Randomly cut out two images from each image.
x_train1_img = load_img ('1.jpg', target_size = (64,64))
x_train1 = img_to_array (x_train1_img)
x_train2_img = load_img ('2.jpg', target_size = (64,64))
x_train2 = img_to_array (x_train2_img)

# (N, 2, 28, 28)->(N, 2, 400)
x_train1 = x_train.reshape (len (x_train1), 2, -1)
x_train2 = x_train.reshape (len (x_train2), 2, -1)
x_test = x_test.reshape (len (x_test), 2, -1)

Convert to # one-hot representation.
y_train = to_categorical (y_train)
y_test = to_categorical (y_test)
#Create a model.
input1 = Input (shape = (1,))
hidden1 = Reshape ((64, 64, 1), input_shape = (64, 64)) (input)
input2 = Input (shape = (1,))
hidden2 = Reshape ((64, 64, 1), input_shape = (64, 64)) (input)
# From input 1 to before joining
x = Dense (1, activation = "relu") (hidden1)
x = Model (inputs = hidden1, outputs = x)
# From input 2 to before joining
y = Dense (1, activation = "relu") (hidden2)
y = Model (inputs = hidden2, outputs = y)
# Join
combined = Concatenate ([x.output, y.output], axis = -1)
# Tightly coupled
z = Dense (32, activation = "relu") (combined)
z = Dense (3, activation = "softmax") (z)
#Model definition and compilation
model = Model (inputs = [input1, input2], outputs = z)
model.compile (loss ='binary_crossentropy', optimizer ='adam', metrics = ['acc'])
model.summary ()
# learn.
history = model.fit ([x_train1, x_train2], y_train, epochs = 40)
What I tried

I searched for the above error, but it didn't work.

  • Answer # 1

    I don't know if the learning accuracy will be correct, but I will show the code that works without error.
    The main questions in the questioner's code are as follows. I fixed that point.

    For some reason, the pre-processing is limited to only two images.

    In the model creation part, the input dimension of the site referenced by the questioner is completely different. It does not apply to the situation of the questioner. By the way, since the questioner is premised on images, I have included a CNN layer.

    As I looked through the code, I referred to an example of a "merge model" in Keras. According to it, I think that the basic pattern is to merge completely different learning results such as images and text. I think that merging equivalent images and images like the questioner is essentially the same as synthesizing images into one and learning with a normal model.

    from keras import backend as K
    from keras.datasets import mnist
    from keras.layers import Activation, Input, Dense, Dropout, Concatenate, Flatten, Conv2D, MaxPool2D
    from keras.models import Model, Sequential
    from keras.utils.np_utils import to_categorical
    import matplotlib.pyplot as plt
    import numpy as np
    import glob
    from PIL import Image
    from keras.utils import np_utils
    from sklearn.model_selection import train_test_split
    from keras.preprocessing.image import load_img, img_to_array
    classes = [
               "Dog", "Cat", "Rabbit"
               ]
    X = []
    Y = []
    for index, classlabel in enumerate (classes):
        dir = "./images/" + classlabel
        files = glob.glob (dir + "/*.jpeg")
        for i, file in enumerate (files):
            image = Image.open (file)
            image = image.convert ("RGB")
            image = image.resize ((64, 64))
            data = np.asarray (image)
            X.append (data)
            Y.append (index)
    X = np.array (X)
    Y = np.array (Y)
    X = X.astype ('float32') /255.0
    # ★★ Fixed pre-processing from here
    #Number of categories
    num_classes = Y.max () --Y.min () + 1
    # Divide image X into x1 and x2 and define a new correct label y
    split_size = len (X) // 2
    x1, x2, y1, y2 = train_test_split (X, Y, test_size = split_size, train_size = split_size)
    # y is made from y1 and y2, which is the average value here. Is the average of dogs and rabbits cats?
    y = (y1 + y2) // 2
    x_train1, x_test1, x_train2, x_test2, y_train, y_test = train_test_split (x1, x2, y, test_size = 0.30)
    # ★★ Corrected so far
    Convert to # one-hot representation.
    y_train = to_categorical (y_train, num_classes)
    y_test = to_categorical (y_test, num_classes)
    # ★★ Modify the model from here for the image
    #Create a model.
    model1 = Sequential ([[
        Conv2D (64,3, input_shape = (64,64,3)),
        Activation ('relu'),
        Conv2D (64,3),
        Activation ('relu'),
        MaxPool2D (pool_size = (2,2)),
        Flatten (),
        Dense (1024),
        Activation ('relu')
    ])
    model2 = Sequential ([[
        Conv2D (64,3, input_shape = (64,64,3)),
        Activation ('relu'),
        Conv2D (64,3),
        Activation ('relu'),
        MaxPool2D (pool_size = (2,2)),
        Flatten (),
        Dense (1024),
        Activation ('relu')
    ])
    concatenated = Concatenate (axis = -1) ([model1.output, model2.output])
    # Tightly coupled
    out = Dense (num_classes, activation ='softmax') (concatenated)
    model = Model ([model1.input, model2.input], out)
    # ★★ Corrected so far
    #Model compilation
    model.compile (loss ='binary_crossentropy', optimizer ='adam', metrics = ['acc'])
    model.summary ()
    # learn.
    history = model.fit ([x_train1, x_train2], y_train, epochs = 40)