{"id":2468,"date":"2025-05-12T13:35:49","date_gmt":"2025-05-12T05:35:49","guid":{"rendered":"https:\/\/thereisno.top\/?p=2468"},"modified":"2025-05-12T13:36:59","modified_gmt":"2025-05-12T05:36:59","slug":"torch-%e5%bf%ab%e9%80%9f%e5%85%a5%e9%97%a8","status":"publish","type":"post","link":"https:\/\/thereisno.top\/?p=2468","title":{"rendered":"torch \u5feb\u901f\u5165\u95e8"},"content":{"rendered":"\n

Quickstart<\/h1>\n\n\n\n

This section runs through the API for common tasks in machine learning. Refer to the links in each section to dive deeper.<\/p>\n\n\n\n

Working with data<\/h2>\n\n\n\n

PyTorch has two primitives to work with data<\/a>: torch.utils.data.DataLoader<\/code>and torch.utils.data.Dataset<\/code>. Dataset<\/code> stores the samples and their corresponding labels, and DataLoader<\/code> wraps an iterable around the Dataset<\/code>.<\/p>\n\n\n\n

<\/a>import<\/strong> torch\n<\/a>from<\/strong> torch import<\/strong> nn\n<\/a>from<\/strong> torch.utils.data import<\/strong> DataLoader\n<\/a>from<\/strong> torchvision import<\/strong> datasets\n<\/a>from<\/strong> torchvision.transforms import<\/strong> ToTensor<\/code><\/pre>\n\n\n\n
PyTorch offers domain-specific libraries such as TorchText<\/a>, TorchVision<\/a>, andTorchAudio<\/a>, all of which include datasets. For this tutorial, we will be using a TorchVision dataset.<\/p>\n\n\n\n
The torchvision.datasets<\/code> module contains Dataset<\/code> objects for many real-world vision data like CIFAR, COCO (full list here<\/a>). In this tutorial, we use the FashionMNIST dataset. Every TorchVision Dataset<\/code> includes two arguments:transform<\/code> and target_transform<\/code> to modify the samples and labels respectively.<\/p>\n\n\n\n
<\/a># Download training data from open datasets.<\/em>\n<\/a>training_data = datasets.FashionMNIST(\n<\/a>    root=\"data\",\n<\/a>    train=True,\n<\/a>    download=True,\n<\/a>    transform=ToTensor(),\n<\/a>)\n<\/a>\n<\/a># Download test data from open datasets.<\/em>\n<\/a>test_data = datasets.FashionMNIST(\n<\/a>    root=\"data\",\n<\/a>    train=False,\n<\/a>    download=True,\n<\/a>    transform=ToTensor(),\n<\/a>)\n<\/a>print(len(training_data))<\/mark><\/code><\/pre>\n\n\n\n
60000<\/mark>\n<\/code><\/pre>\n\n\n\n\n\n\n\n
We pass the Dataset<\/code> as an argument to DataLoader<\/code>. This wraps an iterable over our dataset, and supports automatic batching, sampling, shuffling and multiprocess data loading. Here we define a batch size of 64, i.e. each element in the dataloader iterable will return a batch of 64 features and labels.<\/p>\n\n\n\n
DataLoader<\/code><\/strong> \u8fd4\u56de\u7b2c\u4e00\u9879\u662f\u6570\u636e\uff0c\u7b2c\u4e8c\u9879\u76ee\u662f\u6807\u7b7e<\/mark><\/p>\n\n\n\n
<\/a>batch_size = 64\n<\/a>\n<\/a># Create data loaders.<\/em>\n<\/a>train_dataloader = DataLoader(training_data, batch_size=batch_size)\n<\/a>test_dataloader = DataLoader(test_data, batch_size=batch_size)\n<\/a>\n<\/a>for<\/strong> X, y in<\/strong> test_dataloader:\n<\/a>    print(f\"Shape of X [N, C, H, W]: {X.shape}\")\n<\/a>    print(f\"Shape of y: {y.shape} {y.dtype}\")\n<\/a>    break<\/strong>\n<\/a>\n<\/a>print(len(train_dataloader))<\/mark><\/code><\/pre>\n\n\n\n
Shape of X [N, C, H, W]: torch.Size([64, 1, 28, 28])\nShape of y: torch.Size([64]) torch.int64\n938<\/mark> #\u517160000\u6761\u6570\u636e\uff0c\u6bcf\u4e2a\u6279\u6b2164\uff0c\u4e00\u5171938\u6279\n<\/mark><\/code><\/pre>\n\n\n\n
Read more about loading data in PyTorch<\/a>.<\/p>\n\n\n\n
\n\n\n\n
Creating Models<\/h1>\n\n\n\n
To define a neural network in PyTorch, we create a class that inherits from nn.Module<\/a>. We define the layers of the network in the __init__<\/code> function and specify how data will pass through the network in the forward<\/code> function. To accelerate operations in the neural network, we move it to the accelerator<\/a> such as CUDA, MPS, MTIA, or XPU. If the current accelerator is available, we will use it. Otherwise, we use the CPU.<\/p>\n\n\n\n
<\/a>device = torch.accelerator.current_accelerator().type if<\/strong> torch.accelerator.is_available() else<\/strong> \"cpu\"\n<\/a>\n<\/a>print(f\"Using {device} device\")\n<\/a>\n<\/a># Define model<\/em>\n<\/a>class<\/strong> NeuralNetwork(nn.Module):\n<\/a>    def<\/strong> __init__(self):\n<\/a>        super().__init__()\n<\/a>        self.flatten = nn.Flatten() #\u7ef4\u5ea6\u5c55\u5e73<\/em>\n<\/a>        self.linear_relu_stack = nn.Sequential(\n<\/a>            nn.Linear(28*28, 512),\n<\/a>            nn.ReLU(),\n<\/a>            nn.Linear(512, 512),\n<\/a>            nn.ReLU(),\n<\/a>            nn.Linear(512, 10)\n<\/a>        )\n<\/a>\n<\/a>    def<\/strong> forward(self, x):\n<\/a>        x = self.flatten(x)\n<\/a>        logits = self.linear_relu_stack(x)\n<\/a>        return<\/strong> logits\n<\/a>\n<\/a>model = NeuralNetwork().to(device)\n<\/a>print(model)<\/code><\/pre>\n\n\n\n
Using mps device\nNeuralNetwork(\n  (flatten): Flatten(start_dim=1, end_dim=-1)\n  (linear_relu_stack): Sequential(\n    (0): Linear(in_features=784, out_features=512, bias=True)\n    (1): ReLU()\n    (2): Linear(in_features=512, out_features=512, bias=True)\n    (3): ReLU()\n    (4): Linear(in_features=512, out_features=10, bias=True)\n  )\n)\n<\/code><\/pre>\n\n\n\n
<\/code><\/pre>\n\n\n\n
Read more about building neural networks in PyTorch<\/a>.<\/p>\n\n\n\n
\n\n\n\n
Optimizing the Model Parameters<\/h1>\n\n\n\n
To train a model, we need a loss function<\/a> and an optimizer<\/a>.<\/p>\n\n\n\n
<\/a>loss_fn = nn.CrossEntropyLoss()\n<\/a>optimizer = torch.optim.SGD(model.parameters(), lr=1e-3)<\/code><\/pre>\n\n\n\n
In a single training loop, the model makes predictions on the training dataset (fed to it in batches), and backpropagates the prediction error to adjust the model’s parameters.<\/p>\n\n\n\n
<\/a>def<\/strong> train(dataloader, model, loss_fn, optimizer):\n<\/a>    size = len(dataloader.dataset)\n<\/a>    print(\"size=\"+str(size))\n<\/a>    model.train() # \u542f\u7528 Batch Normalization \u548c Dropout\uff0c\u5f52\u4e00\u5316\uff0c\u968f\u673a\u4e22\u5f03\u795e\u7ecf\u5143\u9632\u6b62\u8fc7\u62df\u5408\uff0c\u6d4b\u8bd5\u65f6\u4e0d\u4e22\u5f03<\/mark><\/em>\n<\/a>    for<\/strong> batch, (X, y) in<\/strong> enumerate(dataloader):\n<\/a>        X, y = X.to(device), y.to(device)\n<\/a>\n<\/a>        # Compute prediction error<\/em>\n<\/a>        pred = model(X)\n<\/a>        #print(\"---------->pred=\")<\/em>\n<\/a>        #print(pred)<\/em>\n<\/a>        #print(\"-----------------------<\")<\/em>\n<\/a>        loss = loss_fn(pred, y)\n<\/a>\n<\/a>        # Backpropagation<\/em>\n<\/a>        loss.backward() # \u8ba1\u7b97\u68af\u5ea6<\/mark><\/em>\n<\/a>        optimizer.step() # \u6839\u636e\u68af\u5ea6\u4f18\u5316\u53c2\u6570<\/mark><\/em>\n<\/a>        optimizer.zero_grad() # \u68af\u5ea6\u5f52\u96f6<\/mark><\/em>\n<\/a>\n<\/a>        if<\/strong> batch % 100 == 0: # \u6bcf100\u4e2abatch\u6253\u5370\u4e00\u6b21<\/mark><\/em>\n<\/a>            loss, current = loss.item(), (batch + 1) * len(X)\n<\/a>            print(f\"loss: {loss:>7f}  [{current:>5d}\/{size:>5d}]\")<\/code><\/pre>\n\n\n\n
We also check the model’s performance against the test dataset to ensure it is learning.<\/p>\n\n\n\n
<\/a>def<\/strong> test(dataloader, model, loss_fn):\n<\/a>    size = len(dataloader.dataset)\n<\/a>    num_batches = len(dataloader)\n<\/a>    model.eval()\n<\/a>    test_loss, correct = 0, 0\n<\/a>    with<\/strong> torch.no_grad():\n<\/a>        for<\/strong> X, y in<\/strong> dataloader:\n<\/a>            X, y = X.to(device), y.to(device)\n<\/a>            pred = model(X)\n<\/a>            test_loss += loss_fn(pred, y).item()\n<\/a>            correct += (pred.argmax(1) == y).type(torch.float).sum().item()\n<\/a>    test_loss \/= num_batches\n<\/a>    correct \/= size\n<\/a>    print(f\"Test Error: \\n Accuracy: {(100*correct):>0.1f}%, Avg loss: {test_loss:>8f} \\n\")<\/code><\/pre>\n\n\n\n
The training process is conducted over several iterations (epochs<\/em>). During each epoch, the model learns parameters to make better predictions. We print the model’s accuracy and loss at each epoch; we’d like to see the accuracy increase and the loss decrease with every epoch.<\/p>\n\n\n\n
<\/a>epochs = 5\n<\/a>for<\/strong> t in<\/strong> range(epochs):\n<\/a>    print(f\"Epoch {t+1}\\n-------------------------------\")\n<\/a>    train(train_dataloader, model, loss_fn, optimizer)\n<\/a>    test(test_dataloader, model, loss_fn)\n<\/a>print(\"Done!\")<\/code><\/pre>\n\n\n\n
Epoch 1\n-------------------------------\nsize=60000 # 938\u4e2a\u6279\u6b21\uff0c\u6bcf\u4e00\u767e\u6253\u5370\u4e00\u6b21<\/mark>\uff0c\u6253\u537010\u6b21\nloss: 2.315621  [   64\/60000]\nloss: 2.297486  [ 6464\/60000]\nloss: 2.284707  [12864\/60000]\nloss: 2.276697  [19264\/60000]\nloss: 2.251865  [25664\/60000]\nloss: 2.229615  [32064\/60000]\nloss: 2.235096  [38464\/60000]\nloss: 2.205865  [44864\/60000]\nloss: 2.203413  [51264\/60000]\nloss: 2.159754  [57664\/60000]\nTest Error: \n Accuracy: 39.3%, Avg loss: 2.162051 \n\nEpoch 2\n-------------------------------\nsize=60000\nloss: 2.175932  [   64\/60000]\nloss: 2.160811  [ 6464\/60000]\nloss: 2.113213  [12864\/60000]\nloss: 2.128377  [19264\/60000]\nloss: 2.065253  [25664\/60000]\nloss: 2.014427  [32064\/60000]\nloss: 2.038499  [38464\/60000]\nloss: 1.966430  [44864\/60000]\nloss: 1.969670  [51264\/60000]\nloss: 1.885340  [57664\/60000]\nTest Error: \n Accuracy: 51.0%, Avg loss: 1.894609 \n\nEpoch 3\n-------------------------------\nsize=60000\nloss: 1.927377  [   64\/60000]\nloss: 1.893924  [ 6464\/60000]\nloss: 1.791221  [12864\/60000]\nloss: 1.828687  [19264\/60000]\nloss: 1.707735  [25664\/60000]\nloss: 1.663736  [32064\/60000]\nloss: 1.677831  [38464\/60000]\nloss: 1.585051  [44864\/60000]\nloss: 1.603040  [51264\/60000]\nloss: 1.495138  [57664\/60000]\nTest Error: \n Accuracy: 59.2%, Avg loss: 1.523530 \n\nEpoch 4\n-------------------------------\nsize=60000\nloss: 1.584166  [   64\/60000]\nloss: 1.549330  [ 6464\/60000]\nloss: 1.415807  [12864\/60000]\nloss: 1.485298  [19264\/60000]\nloss: 1.358027  [25664\/60000]\nloss: 1.356192  [32064\/60000]\nloss: 1.363756  [38464\/60000]\nloss: 1.292613  [44864\/60000]\nloss: 1.323514  [51264\/60000]\nloss: 1.225876  [57664\/60000]\nTest Error: \n Accuracy: 62.5%, Avg loss: 1.258216 \n\nEpoch 5\n-------------------------------\nsize=60000\nloss: 1.329733  [   64\/60000]\nloss: 1.309326  [ 6464\/60000]\nloss: 1.161434  [12864\/60000]\nloss: 1.264519  [19264\/60000]\nloss: 1.133686  [25664\/60000]\nloss: 1.158902  [32064\/60000]\nloss: 1.173635  [38464\/60000]\nloss: 1.112870  [44864\/60000]\nloss: 1.150839  [51264\/60000]\nloss: 1.069254  [57664\/60000]\nTest Error: \n Accuracy: 64.2%, Avg loss: 1.094074 \n\nDone!\n<\/code><\/pre>\n\n\n\n
Read more about Training your model<\/a>.<\/p>\n\n\n\n
\n\n\n\n
Saving Models<\/h1>\n\n\n\n
A common way to save a model is to serialize the internal state dictionary (containing the model parameters).<\/p>\n\n\n\n
<\/a>for<\/strong> var_name in<\/strong> model.state_dict():\n<\/a>    print(var_name, \"\\t\", model.state_dict()[var_name])\n<\/a>\n<\/a>for<\/strong> var_name in<\/strong> optimizer.state_dict():\n<\/a>    print(var_name, \"\\t\", optimizer.state_dict()[var_name])<\/code><\/pre>\n\n\n\n
state \t {}\nparam_groups \t [{'lr': 0.001, 'momentum': 0, 'dampening': 0, 'weight_decay': 0, 'nesterov': False, 'maximize': False, 'foreach': None, 'differentiable': False, 'fused': None, 'params': [0, 1, 2, 3, 4, 5]}]\n<\/code><\/pre>\n\n\n\n
<\/a>torch.save(model.state_dict(), \"model.pth\")\n<\/a>print(\"Saved PyTorch Model State to model.pth\")<\/code><\/pre>\n\n\n\n
Saved PyTorch Model State to model.pth\n<\/code><\/pre>\n\n\n\n
Loading Models<\/h1>\n\n\n\n
The process for loading a model includes re-creating the model structure and loading the state dictionary into it.<\/p>\n\n\n\n
<\/a>model = NeuralNetwork().to(device)\n<\/a>model.load_state_dict(torch.load(\"model.pth\", weights_only=True))<\/code><\/pre>\n\n\n\n
This model can now be used to make predictions.<\/p>\n\n\n\n
<\/a>classes = [\n<\/a>    \"T-shirt\/top\",\n<\/a>    \"Trouser\",\n<\/a>    \"Pullover\",\n<\/a>    \"Dress\",\n<\/a>    \"Coat\",\n<\/a>    \"Sandal\",\n<\/a>    \"Shirt\",\n<\/a>    \"Sneaker\",\n<\/a>    \"Bag\",\n<\/a>    \"Ankle boot\",\n<\/a>]\n<\/a>\n<\/a>model.eval()\n<\/a>x, y = test_data[0][0], test_data[0][1]\n<\/a>with<\/strong> torch.no_grad():\n<\/a>    x = x.to(device)\n<\/a>    pred = model(x)\n<\/a>    predicted, actual = classes[pred[0].argmax(0)], classes[y]\n<\/a>    print(f'Predicted: \"{predicted}\", Actual: \"{actual}\"')<\/code><\/pre>\n\n\n\n
Predicted: \"Ankle boot\", Actual: \"Ankle boot\"\n<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"
Quickstart This section runs through the API for common … <\/p>\n
\u7ee7\u7eed\u9605\u8bfb\u201ctorch \u5feb\u901f\u5165\u95e8\u201d<\/span><\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[246,14],"tags":[253],"class_list":["post-2468","post","type-post","status-publish","format-standard","hentry","category-ai","category-python","tag-torch"],"_links":{"self":[{"href":"https:\/\/thereisno.top\/index.php?rest_route=\/wp\/v2\/posts\/2468","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/thereisno.top\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/thereisno.top\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/thereisno.top\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/thereisno.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=2468"}],"version-history":[{"count":2,"href":"https:\/\/thereisno.top\/index.php?rest_route=\/wp\/v2\/posts\/2468\/revisions"}],"predecessor-version":[{"id":2470,"href":"https:\/\/thereisno.top\/index.php?rest_route=\/wp\/v2\/posts\/2468\/revisions\/2470"}],"wp:attachment":[{"href":"https:\/\/thereisno.top\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=2468"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/thereisno.top\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=2468"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/thereisno.top\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=2468"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}