Pytorch
과제의 파이토치 튜토리얼 부분은 튜토리얼 답게 딱히 설명할 부분은 없어서 코드만 첨부하겠다.
더보기
import torch
def hello():
"""
This is a sample function that we will try to import and run to ensure that
our environment is correctly set up on Google Colab.
"""
print('Hello from pytorch101.py!')
def create_sample_tensor():
"""
Return a torch Tensor of shape (3, 2) which is filled with zeros, except for
element (0, 1) which is set to 10 and element (1, 0) which is set to 100.
Inputs: None
Returns:
- Tensor of shape (3, 2) as described above.
"""
x = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
x = torch.tensor([[0,10],[100,0],[0,0]])
#############################################################################
# END OF YOUR CODE #
#############################################################################
return x
def mutate_tensor(x, indices, values):
"""
Mutate the PyTorch tensor x according to indices and values.
Specifically, indices is a list [(i0, j0), (i1, j1), ... ] of integer indices,
and values is a list [v0, v1, ...] of values. This function should mutate x
by setting:
x[i0, j0] = v0
x[i1, j1] = v1
and so on.
If the same index pair appears multiple times in indices, you should set x to
the last one.
Inputs:
- x: A Tensor of shape (H, W)
- indicies: A list of N tuples [(i0, j0), (i1, j1), ..., ]
- values: A list of N values [v0, v1, ...]
Returns:
- The input tensor x
"""
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
for k in range(len(indices)):
i, j = indices[k]
x[i, j] = values[k]
#############################################################################
# END OF YOUR CODE #
#############################################################################
return x
def count_tensor_elements(x):
"""
Count the number of scalar elements in a tensor x.
For example, a tensor of shape (10,) has 10 elements.a tensor of shape (3, 4)
has 12 elements; a tensor of shape (2, 3, 4) has 24 elements, etc.
You may not use the functions torch.numel or x.numel. The input tensor should
not be modified.
Inputs:
- x: A tensor of any shape
Returns:
- num_elements: An integer giving the number of scalar elements in x
"""
num_elements = None
#############################################################################
# TODO: Implement this function #
# You CANNOT use the built-in functions torch.numel(x) or x.numel(). #
#############################################################################
# Replace "pass" statement with your code
x_shape = x.shape
num_elements = 1
for n in x_shape:
num_elements = num_elements * n
#############################################################################
# END OF YOUR CODE #
#############################################################################
return num_elements
def create_tensor_of_pi(M, N):
"""
Returns a Tensor of shape (M, N) filled entirely with the value 3.14
Inputs:
- M, N: Positive integers giving the shape of Tensor to create
Returns:
- x: A tensor of shape (M, N) filled with the value 3.14
"""
x = None
#############################################################################
# TODO: Implement this function. It should take one line. #
#############################################################################
# Replace "pass" statement with your code
x = torch.full((M,N), 3.14)
#############################################################################
# END OF YOUR CODE #
#############################################################################
return x
def multiples_of_ten(start, stop):
"""
Returns a Tensor of dtype torch.float64 that contains all of the multiples of
ten (in order) between start and stop, inclusive. If there are no multiples
of ten in this range you should return an empty tensor of shape (0,).
Inputs:
- start, stop: Integers with start <= stop specifying the range to create.
Returns:
- x: Tensor of dtype float64 giving multiples of ten between start and stop.
"""
assert start <= stop
x = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
x = []
for i in range(start, stop + 1):
if i%10 == 0:
x.append(i)
x = torch.tensor(x,dtype=torch.float64)
#############################################################################
# END OF YOUR CODE #
#############################################################################
return x
def slice_indexing_practice(x):
"""
Given a two-dimensional tensor x, extract and return several subtensors to
practice with slice indexing. Each tensor should be created using a single
slice indexing operation.
The input tensor should not be modified.
Input:
- x: Tensor of shape (M, N) -- M rows, N columns with M >= 3 and N >= 5.
Returns a tuple of:
- last_row: Tensor of shape (N,) giving the last row of x. It should be a
one-dimensional tensor.
- third_col: Tensor of shape (M, 1) giving the third column of x.
It should be a two-dimensional tensor.
- first_two_rows_three_cols: Tensor of shape (2, 3) giving the data in the
first two rows and first three columns of x.
- even_rows_odd_cols: Two-dimensional tensor containing the elements in the
even-valued rows and odd-valued columns of x.
"""
assert x.shape[0] >= 3
assert x.shape[1] >= 5
last_row = None
third_col = None
first_two_rows_three_cols = None
even_rows_odd_cols = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
last_row = x[-1,:]
third_col = x[:,2]
first_two_rows_three_cols = x[:2,:3]
even_rows_odd_cols = x[::2,1::2]
#############################################################################
# END OF YOUR CODE #
#############################################################################
out = (
last_row,
third_col,
first_two_rows_three_cols,
even_rows_odd_cols,
)
return out
def slice_assignment_practice(x):
"""
Given a two-dimensional tensor of shape (M, N) with M >= 4, N >= 6, mutate its
first 4 rows and 6 columns so they are equal to:
[0 1 2 2 2 2]
[0 1 2 2 2 2]
[3 4 3 4 5 5]
[3 4 3 4 5 5]
Your implementation must obey the following:
- You should mutate the tensor x in-place and return it
- You should only modify the first 4 rows and first 6 columns; all other
elements should remain unchanged
- You may only mutate the tensor using slice assignment operations, where you
assign an integer to a slice of the tensor
- You must use <= 6 slicing operations to achieve the desired result
Inputs:
- x: A tensor of shape (M, N) with M >= 4 and N >= 6
Returns: x
"""
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
x[:4,:6] = torch.tensor([[0,1,2,2,2,2],[0,1,2,2,2,2],[3,4,3,4,5,5],[3,4,3,4,5,5]])
#############################################################################
# END OF YOUR CODE #
#############################################################################
return x
def shuffle_cols(x):
"""
Re-order the columns of an input tensor as described below.
Your implementation should construct the output tensor using a single integer
array indexing operation. The input tensor should not be modified.
Input:
- x: A tensor of shape (M, N) with N >= 3
Returns: A tensor y of shape (M, 4) where:
- The first two columns of y are copies of the first column of x
- The third column of y is the same as the third column of x
- The fourth column of y is the same as the second column of x
"""
y = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
idx = [0,0,2,1]
y = x[:,idx]
#############################################################################
# END OF YOUR CODE #
#############################################################################
return y
def reverse_rows(x):
"""
Reverse the rows of the input tensor.
Your implementation should construct the output tensor using a single integer
array indexing operation. The input tensor should not be modified.
Input:
- x: A tensor of shape (M, N)
Returns: A tensor y of shape (M, N) which is the same as x but with the rows
reversed; that is the first row of y is equal to the last row of x,
the second row of y is equal to the second to last row of x, etc.
"""
y = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
idx = list(reversed([i for i in range(x.shape[0])]))
y = x[idx]
#############################################################################
# END OF YOUR CODE #
#############################################################################
return y
def take_one_elem_per_col(x):
"""
Construct a new tensor by picking out one element from each column of the
input tensor as described below.
The input tensor should not be modified.
Input:
- x: A tensor of shape (M, N) with M >= 4 and N >= 3.
Returns: A tensor y of shape (3,) such that:
- The first element of y is the second element of the first column of x
- The second element of y is the first element of the second column of x
- The third element of y is the fourth element of the third column of x
"""
y = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
idx0 = [1, 0, 3]
idx1 = [0, 1, 2]
y = x[idx0, idx1]
#############################################################################
# END OF YOUR CODE #
#############################################################################
return y
def count_negative_entries(x):
"""
Return the number of negative values in the input tensor x.
Your implementation should perform only a single indexing operation on the
input tensor. You should not use any explicit loops. The input tensor should
not be modified.
Input:
- x: A tensor of any shape
Returns:
- num_neg: Integer giving the number of negative values in x
"""
num_neg = 0
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
num_neg = (x < 0).sum()
#############################################################################
# END OF YOUR CODE #
#############################################################################
return num_neg
def make_one_hot(x):
"""
Construct a tensor of one-hot-vectors from a list of Python integers.
Input:
- x: A list of N integers
Returns:
- y: A tensor of shape (N, C) and where C = 1 + max(x) is one more than the max
value in x. The nth row of y is a one-hot-vector representation of x[n];
In other words, if x[n] = c then y[n, c] = 1; all other elements of y are
zeros. The dtype of y should be torch.float32.
"""
y = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
y = torch.zeros(len(x), max(x) + 1)
for i in range(len(x)):
y[i, x[i]] = 1
#############################################################################
# END OF YOUR CODE #
#############################################################################
return y
def reshape_practice(x):
"""
Given an input tensor of shape (24,), return a reshaped tensor y of shape
(3, 8) such that
y = [
[x[0], x[1], x[2], x[3], x[12], x[13], x[14], x[15]],
[x[4], x[5], x[6], x[7], x[16], x[17], x[18], x[19]],
[x[8], x[9], x[10], x[11], x[20], x[21], x[22], x[23]],
]
You must construct y by performing a sequence of reshaping operations on x
(view, t, transpose, permute, contiguous, reshape, etc). The input tensor
should not be modified.
Input:
- x: A tensor of shape (24,)
Returns:
- y: A reshaped version of x of shape (3, 8) as described above.
"""
y = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
y = x.view(2,3,4).transpose(0,1).contiguous().view(3,8)
#############################################################################
# END OF YOUR CODE #
#############################################################################
return y
def zero_row_min(x):
"""
Return a copy of x, where the minimum value along each row has been set to 0.
For example, if x is:
x = torch.tensor([[
[10, 20, 30],
[ 2, 5, 1]
]])
Then y = zero_row_min(x) should be:
torch.tensor([
[0, 20, 30],
[2, 5, 0]
])
Your implementation should use reduction and indexing operations; you should
not use any explicit loops. The input tensor should not be modified.
Inputs:
- x: Tensor of shape (M, N)
Returns:
- y: Tensor of shape (M, N) that is a copy of x, except the minimum value
along each row is replaced with 0.
"""
y = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
y = x.clone()
min_idx = y.argmin(dim = 1)
y[range(y.shape[0]), min_idx] = 0
#############################################################################
# END OF YOUR CODE #
#############################################################################
return y
def batched_matrix_multiply(x, y, use_loop=True):
"""
Perform batched matrix multiplication between the tensor x of shape (B, N, M)
and the tensor y of shape (B, M, P).
If use_loop=True, then you should use an explicit loop over the batch
dimension B. If loop=False, then you should instead compute the batched
matrix multiply without an explicit loop using a single PyTorch operator.
Inputs:
- x: Tensor of shape (B, N, M)
- y: Tensor of shape (B, M, P)
- use_loop: Whether to use an explicit Python loop.
Hint: torch.stack, bmm
Returns:
- z: Tensor of shape (B, N, P) where z[i] of shape (N, P) is the result of
matrix multiplication between x[i] of shape (N, M) and y[i] of shape
(M, P). It should have the same dtype as x.
"""
z = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
if use_loop == True:
temp = [x[i].mm(y[i]) for i in range(x.shape[0])]
z = torch.stack(temp)
else:
z = x.bmm(y,)
#############################################################################
# END OF YOUR CODE #
#############################################################################
return z
def normalize_columns(x):
"""
Normalize the columns of the matrix x by subtracting the mean and dividing
by standard deviation of each column. You should return a new tensor; the
input should not be modified.
More concretely, given an input tensor x of shape (M, N), produce an output
tensor y of shape (M, N) where y[i, j] = (x[i, j] - mu_j) / sigma_j, where
mu_j is the mean of the column x[:, j].
Your implementation should not use any explicit Python loops (including
list/set/etc comprehensions); you may only use basic arithmetic operations on
tensors (+, -, *, /, **, sqrt), the sum reduction function, and reshape
operations to facilitate broadcasting. You should not use torch.mean,
torch.std, or their instance method variants x.mean, x.std.
Input:
- x: Tensor of shape (M, N).
Returns:
- y: Tensor of shape (M, N) as described above. It should have the same dtype
as the input x.
"""
y = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
mean_x = x.mean(dim=0)
std_x = x.std(dim=0)
y = x - mean_x
y = y / std_x
#############################################################################
# END OF YOUR CODE #
#############################################################################
return y
def mm_on_cpu(x, w):
"""
(helper function) Perform matrix multiplication on CPU.
PLEASE DO NOT EDIT THIS FUNCTION CALL.
Input:
- x: Tensor of shape (A, B), on CPU
- w: Tensor of shape (B, C), on CPU
Returns:
- y: Tensor of shape (A, C) as described above. It should not be in GPU.
"""
y = x.mm(w)
return y
def mm_on_gpu(x, w):
"""
Perform matrix multiplication on GPU
Specifically, you should (i) place each input on GPU first, and then
(ii) perform the matrix multiplication operation. Finally, (iii) return the
final result, which is on CPU for a fair in-place replacement with the mm_on_cpu.
When you move the tensor to GPU, PLEASE use "your_tensor_intance.cuda()" operation.
Input:
- x: Tensor of shape (A, B), on CPU
- w: Tensor of shape (B, C), on CPU
Returns:
- y: Tensor of shape (A, C) as described above. It should not be in GPU.
"""
y = None
#############################################################################
# TODO: Implement this function #
#############################################################################
# Replace "pass" statement with your code
x = x.cuda()
w = w.cuda()
y = x.mm(w)
#############################################################################
# END OF YOUR CODE #
#############################################################################
return y.cpu()
K Nearest Neighbor
K Nearest Neighor는 우선 L2 Distance, 혹은 Euclidean Distance를 구현하는 것부터 시작한다.
각각 반복문을 2번, 1번, 그리고 1개도 사용하지 않고 총 3개의 함수를 구현해야 하는데
반복문을 2번 사용하는 것을 제외하고는 파이썬의 브로드캐스팅기능을 적절히 사용하여야 한다.
평소에 간단한 수준의 브로드캐스팅 기능만 사용해왔던 터라 no-loop 함수 구현할 때 좀 애먹었었다.
그리고 함정도 있었는데, 이 과제는 2020 버전으로 2019 버전에서 몇 가지 개정되었다. 그런데 코드를 어떻게
작성해야 하는지가 나와있는 주석에서 개정된 사항 일부를 반영하지 않아서 이것 때문에 시간을 많이 뺏겼었다.
Distance를 구하는 함수에서 데이터는 항상 C× H× W shape으로 주어진다고 했는데, 나중에 사용될 때는
그게 아니었다 ㅜㅜ 그래서 no_loop 함수는 shape 처리를 따로 해주어야만 했다.
더보기
def compute_distances_two_loops(x_train, x_test):
"""
Computes the squared Euclidean distance between each element of the training
set and each element of the test set. Images should be flattened and treated
as vectors.
This implementation uses a naive set of nested loops over the training and
test data.
The input data may have any number of dimensions -- for example this function
should be able to compute nearest neighbor between vectors, in which case
the inputs will have shape (num_{train, test}, D); it should alse be able to
compute nearest neighbors between images, where the inputs will have shape
(num_{train, test}, C, H, W). More generally, the inputs will have shape
(num_{train, test}, D1, D2, ..., Dn); you should flatten each element
of shape (D1, D2, ..., Dn) into a vector of shape (D1 * D2 * ... * Dn) before
computing distances.
The input tensors should not be modified.
NOTE: Your implementation may not use `torch.norm`, `torch.dist`,
`torch.cdist`, or their instance method variants x.norm / x.dist / x.cdist.
You may not use any functions from torch.nn or torch.nn.functional.
Inputs:
- x_train: Torch tensor of shape (num_train, D1, D2, ...)
- x_test: Torch tensor of shape (num_test, D1, D2, ...)
Returns:
- dists: Torch tensor of shape (num_train, num_test) where dists[i, j] is the
squared Euclidean distance between the ith training point and the jth test
point. It should have the same dtype as x_train.
"""
# Initialize dists to be a tensor of shape (num_train, num_test) with the
# same datatype and device as x_train
num_train = x_train.shape[0]
num_test = x_test.shape[0]
dists = x_train.new_zeros(num_train, num_test)
##############################################################################
# TODO: Implement this function using a pair of nested loops over the #
# training data and the test data. #
# #
# You may not use torch.norm (or its instance method variant), nor any #
# functions from torch.nn or torch.nn.functional. #
##############################################################################
# Replace "pass" statement with your code
for i in range(num_train):
for j in range(num_test):
dists[i][j] = (x_train[i] - x_test[j]).pow(2).sum().sqrt()
##############################################################################
# END OF YOUR CODE #
##############################################################################
return dists
def compute_distances_one_loop(x_train, x_test):
"""
Computes the squared Euclidean distance between each element of the training
set and each element of the test set. Images should be flattened and treated
as vectors.
This implementation uses only a single loop over the training data.
Similar to compute_distances_two_loops, this should be able to handle inputs
with any number of dimensions. The inputs should not be modified.
NOTE: Your implementation may not use `torch.norm`, `torch.dist`,
`torch.cdist`, or their instance method variants x.norm / x.dist / x.cdist.
You may not use any functions from torch.nn or torch.nn.functional.
Inputs:
- x_train: Torch tensor of shape (num_train, D1, D2, ...)
- x_test: Torch tensor of shape (num_test, D1, D2, ...)
Returns:
- dists: Torch tensor of shape (num_train, num_test) where dists[i, j] is the
squared Euclidean distance between the ith training point and the jth test
point.
"""
# Initialize dists to be a tensor of shape (num_train, num_test) with the
# same datatype and device as x_train
num_train = x_train.shape[0]
num_test = x_test.shape[0]
dists = x_train.new_zeros(num_train, num_test)
##############################################################################
# TODO: Implement this function using only a single loop over x_train. #
# #
# You may not use torch.norm (or its instance method variant), nor any #
# functions from torch.nn or torch.nn.functional. #
##############################################################################
# Replace "pass" statement with your code
x_train = x_train.view(num_train, x_train.shape[1] * x_train.shape[2] * x_train.shape[3])
x_test = x_test.view(num_test, x_test.shape[1] * x_test.shape[2] * x_test.shape[3])
for i in range(num_train):
dists[i] = (x_train[i] - x_test).pow(2).sum(dim = 1).sqrt()
##############################################################################
# END OF YOUR CODE #
##############################################################################
return dists
def compute_distances_no_loops(x_train, x_test):
"""
Computes the squared Euclidean distance between each element of the training
set and each element of the test set. Images should be flattened and treated
as vectors.
This implementation should not use any Python loops. For memory-efficiency,
it also should not create any large intermediate tensors; in particular you
should not create any intermediate tensors with O(num_train*num_test)
elements.
Similar to compute_distances_two_loops, this should be able to handle inputs
with any number of dimensions. The inputs should not be modified.
NOTE: Your implementation may not use `torch.norm`, `torch.dist`,
`torch.cdist`, or their instance method variants x.norm / x.dist / x.cdist.
You may not use any functions from torch.nn or torch.nn.functional.
Inputs:
- x_train: Torch tensor of shape (num_train, C, H, W)
- x_test: Torch tensor of shape (num_test, C, H, W)
Returns:
- dists: Torch tensor of shape (num_train, num_test) where dists[i, j] is the
squared Euclidean distance between the ith training point and the jth test
point.
"""
# Initialize dists to be a tensor of shape (num_train, num_test) with the
# same datatype and device as x_train
num_train = x_train.shape[0]
num_test = x_test.shape[0]
dists = x_train.new_zeros(num_train, num_test)
##############################################################################
# TODO: Implement this function without using any explicit loops and without #
# creating any intermediate tensors with O(num_train * num_test) elements. #
# #
# You may not use torch.norm (or its instance method variant), nor any #
# functions from torch.nn or torch.nn.functional. #
# #
# HINT: Try to formulate the Euclidean distance using two broadcast sums #
# and a matrix multiply. #
##############################################################################
# Replace "pass" statement with your code
shape = torch.tensor(x_train.shape[1:])
s = torch.prod(shape)
x_train = x_train.view(num_train, s)
x_test = x_test.view(num_test, s)
xx_train = x_train.pow(2).sum(dim=1)
xx_test = x_test.pow(2).sum(dim=1)
xx_mm = x_train.mm(x_test.t())
dists = (xx_train.reshape(-1,1) + xx_test - 2 * xx_mm).sqrt()
##############################################################################
# END OF YOUR CODE #
##############################################################################
return dists
이어서 앞서 구현한 L2 Distance함수를 이용해 test data의 distance를 구하고 이를 이용해 label을 매핑하는 함수를
구현해야 한다. 힌트로 torch.topk 함수를 이용해라고 하는데, 힌트대로 topk 함수를 이용해서 training data에서
k 개의 가장 가까운 k개의 점을 찾은 후, 가장 많이 나온 label을 test data에 매핑하면 무난히 풀 수 있을 것이다.
더보기
def predict_labels(dists, y_train, k=1):
"""
Given distances between all pairs of training and test samples, predict a
label for each test sample by taking a **majority vote** among its k nearest
neighbors in the training set.
In the event of a tie, this function **should** return the smallest label. For
example, if k=5 and the 5 nearest neighbors to a test example have labels
[1, 2, 1, 2, 3] then there is a tie between 1 and 2 (each have 2 votes), so
we should return 1 since it is the smallest label.
This function should not modify any of its inputs.
Inputs:
- dists: Torch tensor of shape (num_train, num_test) where dists[i, j] is the
squared Euclidean distance between the ith training point and the jth test
point.
- y_train: Torch tensor of shape (num_train,) giving labels for all training
samples. Each label is an integer in the range [0, num_classes - 1]
- k: The number of nearest neighbors to use for classification.
Returns:
- y_pred: A torch int64 tensor of shape (num_test,) giving predicted labels
for the test data, where y_pred[j] is the predicted label for the jth test
example. Each label should be an integer in the range [0, num_classes - 1].
"""
num_train, num_test = dists.shape
y_pred = torch.zeros(num_test, dtype=torch.int64)
##############################################################################
# TODO: Implement this function. You may use an explicit loop over the test #
# samples. Hint: Look up the function torch.topk #
##############################################################################
# Replace "pass" statement with your code
for i in range(num_test):
values, indices = dists[:,i].topk(k=k, largest = False)
classes, counts = y_train[indices].unique(return_counts=True)
y_pred[i] = classes[counts.argmax()]
##############################################################################
# END OF YOUR CODE #
##############################################################################
return y_pred
이제는 앞서 만든 함수들을 이용해서 K-Nearest Neighbor Classifier 클래스를 구현해야 한다. 클래스는 trian 과정이라고 할 수 있는 __init__ 메소드, 이름처럼 predict 과정이라고 할 수 있는 predict 메소드, 그리고 predict한 결과의 정확도를 구하는 check_accuracy 메소드로 이루어져 있으며, 이중 check_accuracy는 이미 구현되어 있으니 우리가 해야 할 것은
__init__과 predcit만 구현하면 된다. __init__의 경우 복잡할 것 없이 train data를 저장시키기만 하면 된다. 또한 predict
메소드의 경우도 앞서 구현한 메소드로 거리를 구하고, label을 predict 해주면 끝이 난다.
더보기
class KnnClassifier:
def __init__(self, x_train, y_train):
"""
Create a new K-Nearest Neighbor classifier with the specified training data.
In the initializer we simply memorize the provided training data.
Inputs:
- x_train: Torch tensor of shape (num_train, C, H, W) giving training data
- y_train: int64 torch tensor of shape (num_train,) giving training labels
"""
###########################################################################
# TODO: Implement the initializer for this class. It should perform no #
# computation and simply memorize the training data. #
###########################################################################
# Replace "pass" statement with your code
self.x_train = x_train.contiguous()
self.y_train = y_train.contiguous()
###########################################################################
# END OF YOUR CODE #
###########################################################################
def predict(self, x_test, k=1):
"""
Make predictions using the classifier.
Inputs:
- x_test: Torch tensor of shape (num_test, C, H, W) giving test samples
- k: The number of neighbors to use for predictions
Returns:
- y_test_pred: Torch tensor of shape (num_test,) giving predicted labels
for the test samples.
"""
y_test_pred = None
###########################################################################
# TODO: Implement this method. You should use the functions you wrote #
# above for computing distances (use the no-loop variant) and to predict #
# output labels.
###########################################################################
# Replace "pass" statement with your code
dists = compute_distances_no_loops(self.x_train, x_test.contiguous())
y_test_pred = predict_labels(dists, self.y_train, k=k)
###########################################################################
# END OF YOUR CODE #
###########################################################################
return y_test_pred
def check_accuracy(self, x_test, y_test, k=1, quiet=False):
"""
Utility method for checking the accuracy of this classifier on test data.
Returns the accuracy of the classifier on the test data, and also prints a
message giving the accuracy.
Inputs:
- x_test: Torch tensor of shape (num_test, C, H, W) giving test samples
- y_test: int64 torch tensor of shape (num_test,) giving test labels
- k: The number of neighbors to use for prediction
- quiet: If True, don't print a message.
Returns:
- accuracy: Accuracy of this classifier on the test data, as a percent.
Python float in the range [0, 100]
"""
y_test_pred = self.predict(x_test, k=k)
num_samples = x_test.shape[0]
num_correct = (y_test == y_test_pred).sum().item()
accuracy = 100.0 * num_correct / num_samples
msg = (f'Got {num_correct} / {num_samples} correct; '
f'accuracy is {accuracy:.2f}%')
if not quiet:
print(msg)
return accuracy
마지막으로는 K-Nearest Neighbor의 Hyperparameter인 k를 선정하는 함수를 구현해야 한다.
먼저 cross validation을 수행하는 knn_cross_validate 함수를 구현하고 cross validation의 결과에 따라
최적의 k 값을 선정하는 knn_get_best_k 함수를 구현해야 하는데 강의에서 설명한대로 하면 금방 끝이 난다.
knn_corss_validate 함수는 데이터를 n개의 데이터로 쪼개고 각각 validation을 다르게 선정하면서 테스트 해본다. 그리고 이 과정을 주어진 k 값에 따라서도 각각 테스트 해보고 그 결과를 저장 후 리턴하며, knn_get_best_k 함수는 앞서의
결과들 중에서 가장 결과가 좋았던 k 값을 선정해 리턴하면 된다.
더보기
def knn_cross_validate(x_train, y_train, num_folds=5, k_choices=None):
"""
Perform cross-validation for KnnClassifier.
Inputs:
- x_train: Tensor of shape (num_train, C, H, W) giving all training data
- y_train: int64 tensor of shape (num_train,) giving labels for training data
- num_folds: Integer giving the number of folds to use
- k_choices: List of integers giving the values of k to try
Returns:
- k_to_accuracies: Dictionary mapping values of k to lists, where
k_to_accuracies[k][i] is the accuracy on the ith fold of a KnnClassifier
that uses k nearest neighbors.
"""
if k_choices is None:
# Use default values
k_choices = [1, 3, 5, 8, 10, 12, 15, 20, 50, 100]
# First we divide the training data into num_folds equally-sized folds.
x_train_folds = []
y_train_folds = []
##############################################################################
# TODO: Split the training data and images into folds. After splitting, #
# x_train_folds and y_train_folds should be lists of length num_folds, where #
# y_train_folds[i] is the label vector for images in x_train_folds[i]. #
# Hint: torch.chunk #
##############################################################################
# Replace "pass" statement with your code
x_train_folds = torch.chunk(x_train, num_folds)
y_train_folds = torch.chunk(y_train, num_folds)
##############################################################################
# END OF YOUR CODE #
##############################################################################
# A dictionary holding the accuracies for different values of k that we find
# when running cross-validation. After running cross-validation,
# k_to_accuracies[k] should be a list of length num_folds giving the different
# accuracies we found when trying KnnClassifiers that use k neighbors.
k_to_accuracies = {}
##############################################################################
# TODO: Perform cross-validation to find the best value of k. For each value #
# of k in k_choices, run the k-nearest-neighbor algorithm num_folds times; #
# in each case you'll use all but one fold as training data, and use the #
# last fold as a validation set. Store the accuracies for all folds and all #
# values in k in k_to_accuracies. HINT: torch.cat #
##############################################################################
# Replace "pass" statement with your code
for k in k_choices:
res = []
for i in range(num_folds):
if i == 0:
x_train = torch.cat(x_train_folds[1:])
y_train = torch.cat(y_train_folds[1:])
elif i != num_folds - 1:
x_train = torch.cat((torch.cat(x_train_folds[:i]), torch.cat(x_train_folds[i + 1:])))
y_train = torch.cat((torch.cat(y_train_folds[:i]), torch.cat(y_train_folds[i + 1:])))
else:
x_train = torch.cat(x_train_folds[:i])
y_train = torch.cat(y_train_folds[:i])
x_test = x_train_folds[i]
y_test = y_train_folds[i]
classifier = KnnClassifier(x_train, y_train)
res.append(classifier.check_accuracy(x_test, y_test, k=k))
k_to_accuracies[k] = res
##############################################################################
# END OF YOUR CODE #
##############################################################################
return k_to_accuracies
def knn_get_best_k(k_to_accuracies):
"""
Select the best value for k, from the cross-validation result from
knn_cross_validate. If there are multiple k's available, then you SHOULD
choose the smallest k among all possible answer.
Inputs:
- k_to_accuracies: Dictionary mapping values of k to lists, where
k_to_accuracies[k][i] is the accuracy on the ith fold of a KnnClassifier
that uses k nearest neighbors.
Returns:
- best_k: best (and smallest if there is a conflict) k value based on
the k_to_accuracies info
"""
best_k = 0
##############################################################################
# TODO: Use the results of cross-validation stored in k_to_accuracies to #
# choose the value of k, and store the result in best_k. You should choose #
# the value of k that has the highest mean accuracy accross all folds. #
##############################################################################
# Replace "pass" statement with your code
best_mean = 0
for k in k_to_accuracies.keys():
m = torch.tensor(k_to_accuracies[k]).mean()
if m > best_mean:
best_mean = m
best_k = k
##############################################################################
# END OF YOUR CODE #
##############################################################################
return best_k728x90
'Deep Learning for Computer Vision' 카테고리의 다른 글
| EECS 498-007 / 598-005 Lecture 4 : Optimization (0) | 2021.01.11 |
|---|---|
| EECS 498-007 / 598-005 Lecture 3 : Linear Classifier (2) | 2021.01.07 |
| EECS 498-007 / 598-005 Lecture 2 : Image Classification (0) | 2020.12.30 |
| EECS 498-007 / 598-005 Lecture 1 : Deep Learning for Computer Vision (0) | 2020.12.27 |
| EECS 498-007 / 598-005 시작! (0) | 2020.12.26 |
