用于LSTM输入的熊猫数据成形
用于LSTM输入的熊猫数据成形
提问于 2021-07-23 10:10:53
我有一个简单的数据集，只有两列(年份和油价)。现在，我需要对它们进行造型，以便keras的LSTM层能够接受它们的input_shape。
我的代码如下所示，我基本上需要黄色标记的区域的帮助。我认为我需要改变/转换X_train和X_test之前(数组，规范化等)。但我只有尝试时才会犯错误..。
回答 2
Stack Overflow用户
回答已采纳
发布于 2021-07-24 04:43:06
我认为如果您将X_train和X_test保持为二维数据帧，您的代码就能工作。所以如果你定义
X_train = train[["Year"]]
X_test = test[["Year"]]之后，您可以定义LSTM体系结构，就像您在问题中所做的那样。
Stack Overflow用户
发布于 2021-07-25 06:04:02
一种解决方案是重塑类似于X_train的X_train.reshape((X_train.shape[0], 1, 1))，并在第一个LSTM层中省略input_shape参数。LSTM层的输入形状总是(批量大小、时间步长、特性)。请参阅有关此这里的更多信息。
另一件要考虑的事情是在“小”范围(如0，1)内缩放数据，这样训练过程就能很好地顺利地收敛，因为我们不希望权重在更新时变得疯狂，但它也取决于具体的实现/应用程序。
您可能需要工作旋钮(超参数，如激活函数、抛出、单位、批大小等，以获得更好的预测性能)。
下面是一个完整的工作示例，其中包含了一些评论：
# imports
import numpy as np
import pandas as pd
from sklearn.preprocessing import MinMaxScaler
from keras.models import Sequential
from keras.layers import Dense, LSTM, Dropout
from keras.activations import relu
# data setup
years = [i for i in range(1861, 2021)]
oil = [0.49, 1.05, 3.15, 8.06, 6.59, 3.74, 2.41, 3.63, 3.64, 3.86, 4.34, 3.64, 1.83, 1.17, 1.35, 2.56, 2.42, 1.19, 0.86, 0.95, 0.86, 0.78, 1, 0.84, 0.88, 0.71, 0.67, 0.88, 0.94, 0.87, 0.67, 0.56, 0.64, 0.84, 1.36, 1.18, 0.79, 0.91, 1.29, 1.19, 0.96, 0.8, 0.94, 0.86, 0.62, 0.73, 0.72, 0.72, 0.7, 0.61, 0.61, 0.74, 0.95, 0.81, 0.64, 1.1, 1.56, 1.98, 2.01, 3.07, 1.73, 1.61, 1.34, 1.43, 1.68, 1.88, 1.3, 1.17, 1.27, 1.19, 0.65, 0.87, 0.67, 1, 0.97, 1.09, 1.18, 1.13, 1.02, 1.02, 1.14, 1.19, 1.2, 1.21, 1.05, 1.12, 1.9, 1.99, 1.78, 1.71, 1.71, 1.71, 1.93, 1.93, 1.93, 1.93, 1.9, 2.08, 2.08, 1.9, 1.8, 1.8, 1.8, 1.8, 1.8, 1.8, 1.8, 1.8, 1.8, 1.8, 2.24, 2.48, 3.29, 11.58, 11.53, 12.8, 13.92, 14.02, 31.61, 36.83, 35.93, 32.97, 29.55, 28.78, 27.56, 14.43, 18.43503937, 14.9238417, 18.22611328, 23.72582031, 20.0009144, 19.32083658, 16.97163424, 15.81762646, 17.01667969, 20.66848837, 19.09258755, 12.71566148, 17.97007782, 28.49544922, 24.44389105, 25.02325581, 28.83070313, 38.265, 54.52108949, 65.1440625, 72.38907843, 97.25597276, 61.67126482, 79.4955336, 111.2555976, 111.6697024, 108.6585178, 98.94600791, 52.38675889, 43.73416996, 54.19244048, 71.31005976, 64.21057312, 41.83834646]
data = pd.DataFrame(np.vstack((years, oil)).T, columns = ["Year", "Oil Crude Price ($)"]).astype({'Year': int})
# train percentage, thus test percentage = 1 - train_split
train_split = 0.8
# scaler for inputs and outputs
scaler = MinMaxScaler()
# scaling data between 0 and 1
data_scaled = scaler.fit_transform(data.values)
# splitting data into train set 0.6 * 160 = first 96 rows
X_train = data_scaled[:int(train_split * len(data_scaled)),0]
y_train = data_scaled[:int(train_split * len(data_scaled)),1]
# splitting data into test set 0.4 * 160 = last 64 rows
X_test = data_scaled[int(train_split * len(data_scaled)):,0]
y_test = data_scaled[int(train_split * len(data_scaled)):,1]
# sanity check, adding rows in X_train and X_test MUST add to total rows in data
assert len(X_train) + len(X_test) == len(data)
# reshaping inputs for LSTM
X_train_lstm = X_train.reshape((X_train.shape[0], 1, 1))
X_test_lstm = X_test.reshape((X_test.shape[0], 1, 1))
# building model with several LSTM, dropouts, and dense layers
model = Sequential()
model.add(LSTM(units = 512, return_sequences = True))
model.add(Dropout(0.2))
model.add(LSTM(units = 128, return_sequences = True))
model.add(Dropout(0.2))
model.add(LSTM(units = 64, return_sequences = True))
model.add(Dropout(0.2))
model.add(LSTM(units = 32, return_sequences = True))
model.add(Dropout(0.2))
model.add(LSTM(units = 16))
model.add(Dropout(0.2))
model.add(Dense(units = 1))
# compiling model with rmsprop (my preferred optimizer, and loss)
model.compile(optimizer="adam", loss="mse")
# training model for 500 epocs and 40 samples per batch
history = model.fit(X_train_lstm, y_train, epochs=100, batch_size = 20, verbose=1)
# making predictions using test set
y_pred_scaled = model.predict(X_test_lstm)
def original_scale(scaler, x, y):
return scaler.inverse_transform(np.concatenate((x.reshape((x.shape[0], 1)), y), axis=1))
# transforming values back to original scale
y_pred = original_scale(scaler, X_test, y_pred_scaled)[:,1] # predicted price
y_test = data.values[int(train_split * data_scaled.shape[0]):,1] #
y_test_years = data.values[int(train_split * data_scaled.shape[0]):,0]
# wrapping up putting results together in a dataframe
output = pd.DataFrame(data = np.vstack((y_test_years, y_test, y_pred)).T, columns = ["Year", "Oil Crude Price ($)", "Predicted Oil Crude Price ($)"]).astype({'Year': int})
print(output)输出：
Year Oil Crude Price ($) Predicted Oil Crude Price ($)
0 1989 18.226113 22.428251
1 1990 23.725820 23.613170
2 1991 20.000914 24.847811
3 1992 19.320837 26.132980
4 1993 16.971634 27.469316
5 1994 15.817626 28.857338
6 1995 17.016680 30.297413
7 1996 20.668488 31.789748
8 1997 19.092588 33.334372
9 1998 12.715661 34.931136
10 1999 17.970078 36.579701
11 2000 28.495449 38.279516
12 2001 24.443891 40.029859
13 2002 25.023256 41.829785
14 2003 28.830703 43.678103
15 2004 38.265000 45.573485
16 2005 54.521089 47.514343
17 2006 65.144063 49.498909
18 2007 72.389078 51.525253
19 2008 97.255973 53.591203
20 2009 61.671265 55.694481
21 2010 79.495534 57.832594
22 2011 111.255598 60.002952
23 2012 111.669702 62.202827
24 2013 108.658518 64.429348
25 2014 98.946008 66.679673
26 2015 52.386759 68.950699
27 2016 43.734170 71.239459
28 2017 54.192440 73.542838
29 2018 71.310060 75.857814
30 2019 64.210573 78.181272
31 2020 41.838346 80.510184包版本：
keras 2.4.3
numpy 1.19.2
pandas 1.1.5
scikit-learn 0.23.2
tensorflow 2.4.1页面原文内容由Stack Overflow提供。腾讯云小微IT领域专用引擎提供翻译支持
原文链接：
https://stackoverflow.com/questions/68503489
复制相关文章
相似问题
添加站长 进交流群
领取专属 10元无门槛券
AI混元助手 在线答疑
关注 腾讯云开发者公众号
洞察 腾讯核心技术
剖析业界实践案例
腾讯云开发者
