{"id":1821,"date":"2022-08-16T11:48:12","date_gmt":"2022-08-16T10:48:12","guid":{"rendered":"https:\/\/wp.coventry.domains\/e2edu\/?page_id=1821"},"modified":"2022-08-24T16:02:37","modified_gmt":"2022-08-24T15:02:37","slug":"image-generation-with-a-gan","status":"publish","type":"page","link":"https:\/\/wp.coventry.domains\/e2edu\/image-generation-with-a-gan\/","title":{"rendered":"Image Generation with a GAN"},"content":{"rendered":"\n

Summary<\/h2>\n\n\n\n

The following tutorial introduces the use of a simple generative adversarial network for generating synthetic images. This model can be trained on a collection of images. The model uses a combination of a conventional neural network (ANN) and a convolutional neural network (CNN) to create images.<\/p>\n\n\n\n

This tutorial forms part of a series of tutorials on using PyTorch to create and train generative deep learning models. The code for these tutorials is available here<\/a>.<\/p>\n\n\n\n

In this example, the model is trained on images that have been collected on Flickr by using a full text search for the words “dancer, dance, contemporary, solo”. <\/p>\n\n\n\n

After training the model on about 4000 images for up to 1000 epochs, it generates images such as these.<\/p>\n\n\n

\n
\"\"
Synthetic Images of Dancers Created after 447 Epochs of Training.<\/figcaption><\/figure>\n<\/div>\n\n
\n
\"\"
Synthetic Images of Dancers Created after 576 Epochs of Training.<\/figcaption><\/figure>\n<\/div>\n\n
\n
\"\"
Synthetic Images of Dancers Created after 959 Epochs of Training.<\/figcaption><\/figure>\n<\/div>\n\n\n

Imports<\/h2>\n\n\n\n

The following modules need to be available and imported for this example. <\/p>\n\n\n\n

import numpy as np\nimport torch\nimport torchvision\nfrom pytorch_model_summary import summary\nfrom torch.utils.data import DataLoader\nfrom torch import nn\nfrom torch import optim\nfrom collections import OrderedDict\nimport matplotlib.pyplot as plt\nimport time\nimport pickle\nimport math<\/code><\/pre>\n\n\n\n
Compute Device<\/h2>\n\n\n\n
If a GPU is available for running the model, this device can be selected as follows.<\/p>\n\n\n\n
device = 'cuda' if torch.cuda.is_available() else 'cpu'\nprint('Using {} device'.format(device))<\/code><\/pre>\n\n\n\n
Import Images and Create Dataset<\/h2>\n\n\n\n
The easiest method to import images that are stored in local directories involves the use of the ImageFolder class that forms part of the torchvision.datasets submodule. This class represents a Dataset for images that can be directly used for creating Dataloaders. This avoids having to write your own code for importing images from directories, converting them into tensors, and implementing a custom Dataset class. The constructor of the ImageFolder class takes two arguments: the top level path to where all images are stored and an instance of the torchvision.transforms.Compose class. <\/p>\n\n\n\n
The path provided to the constructor must be one level higher in the directory structure than the directory or directories that contain the images. The reason for this is that the distribution of images across directories is interpreted as classification of the images, where each directory name represents the class label for the images that are contained within it. Since we don’t need class labels for this example, it is fine to store all images in a single directory. Here, this single directory has the name “Flickr_Dancers” and contains approximately 4000 images of dancers and dance settings that have been downloaded from Flickr. The downloaded images have a resolution of 150×150 pixels and contain three colour channels. The Python code used for scraping images from Flickr is described here<\/a>. The already downloaded images are available here<\/a>.<\/p>\n\n\n\n
The instance of the Compose class which forms part of the torchvision.transforms submodule provides the functionality to process images in the dataset before they are passed as input data into a machine learning model. The constructor of the Compose class takes as argument a list of instances of classes for image processing. Several such classes are provided by the torchvision.transforms submodule. In the code example, only two instances for image processing are passed to the constructor, one for resizing the images and one for converting the images into tensors. Once the Compose class has been instantiated, it will automatically process images in the sequence in which the image processing instances were passed to the constructor. <\/p>\n\n\n\n
The code for creating a Dataset for images is as follows:<\/p>\n\n\n\n
image_data_path = \"..\/..\/data\/Images\"\nimage_size = 128\n\ntransform = torchvision.transforms.Compose([ torchvision.transforms.Resize(image_size), torchvision.transforms.ToTensor() ])\n\nfull_dataset = torchvision.datasets.ImageFolder(image_data_path, transform=transform)<\/code><\/pre>\n\n\n\n
This dataset contains all data. The dataset can be split into two datasets, one for training and one for testing, as follows:<\/p>\n\n\n\n
test_percentage  = 0.2\n\ndataset_size = len(full_dataset)\n\ntest_size = int(test_percentage * dataset_size)\ntrain_size = dataset_size - test_size\n\ntrain_dataset, test_dataset = torch.utils.data.random_split(full_dataset, [train_size, test_size])<\/code><\/pre>\n\n\n\n
DataLoaders are created from these two datasets as follows:<\/p>\n\n\n\n
batch_size = 16\n\ntrain_dataloader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)\ntest_dataloader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)<\/code><\/pre>\n\n\n\n
Create Models<\/h2>\n\n\n\n
A generative adversarial network (GAN) consists of two models. One for generating synthetic data from input noise, and the other for discriminating between original data and synthetic data. The former is named “Generator” and the latter “Critique”. A brief introduction to GANs is available here<\/a>. In this example, the two models operate on images. The Generator model takes as input a tensor containing noise and produces as output a tensor representing synthetic images. The noise vector represents the encoding of the synthetic image. The Critique takes as input a tensor representing images and produces as output a tensor that classifies the images as ether real or fake (synthetic). Both models employ as networks a combination of convolutional neural networks (CNN) and conventional artificial neural networks (ANN). <\/p>\n\n\n\n
Create Critique Model<\/h3>\n\n\n\n
For converting input images into output classes, the Critique model first passes the input images through several convolution layers, then flattens the feature map that is output by the last convolution layer into a one-dimensional feature vector. This feature vector is then passed through several conventional neural network layers with the last layer outputting a single value. <\/p>\n\n\n\n
The CNN part of the model successively reduces the size of the feature maps while increasing their number of channels. Each convolution layer is followed by a leaky RelU activation function and a batch normalisation. <\/p>\n\n\n\n
The ANN part of the model successively reduces the dimension of the one-dimensional feature vector down to 1. Each ANN layer with the exception of the last one is also followed by a leaky ReLU activation function. The  last ANN layer is followed by a Softmax activation function to obtain normalised class probabilities. <\/p>\n\n\n\n
The class definition of the Critique model is as follows:<\/p>\n\n\n\n
class Critique(nn.Module):\n    def __init__(self, image_size, image_channels, conv_channel_counts, conv_kernel_size, dense_layer_sizes):\n        super().__init__()\n        \n        self.image_size = image_size\n        self.image_channels = image_channels\n        self.conv_channel_counts = conv_channel_counts\n        self.conv_kernel_size = conv_kernel_size\n        self.dense_layer_sizes = dense_layer_sizes\n        \n        # create convolutional layers\n        conv_layers = []\n        \n        stride = (self.conv_kernel_size - 1) \/\/ 2\n        padding = stride\n        \n        conv_layers.append((\"critique_conv_0\", nn.Conv2d(image_channels, self.conv_channel_counts[0], self.conv_kernel_size, stride=stride, padding=padding)))\n        conv_layers.append((\"critique_lrelu_0\", nn.LeakyReLU(0.2)))\n        conv_layers.append((\"critique_bnorm_0\", nn.BatchNorm2d(self.conv_channel_counts[0])))\n        \n        conv_layer_count = len(conv_channel_counts)\n        \n        for layer_index in range(1, conv_layer_count):\n            conv_layers.append((\"critique_conv_{}\".format(layer_index), nn.Conv2d(self.conv_channel_counts[layer_index-1], self.conv_channel_counts[layer_index], self.conv_kernel_size, stride=stride, padding=padding)))\n            conv_layers.append((\"critique_lrelu_{}\".format(layer_index), nn.LeakyReLU(0.2)))\n            conv_layers.append((\"critique_bnorm_{}\".format(layer_index), nn.BatchNorm2d(self.conv_channel_counts[layer_index])))\n\n        self.conv_layers = nn.Sequential(OrderedDict(conv_layers))\n        self.flatten = nn.Flatten(start_dim=1)\n        \n        # create dense layers\n        dense_layers = []\n        \n        last_conv_layer_size = image_size \/\/ np.power(2, len(conv_channel_counts))\n        \n        #print(\"last_conv_layer_size \", last_conv_layer_size)\n        \n        dense_layer_input_size = conv_channel_counts[-1] * last_conv_layer_size * last_conv_layer_size\n        \n        #print(\"dense_layer_input_size \", dense_layer_input_size)\n        \n        dense_layers.append((\"critique_dense_0\", nn.Linear(dense_layer_input_size, self.dense_layer_sizes[0])))\n        dense_layers.append((\"critique_dense_lrelu_0\", nn.LeakyReLU(0.2)))\n        \n        dense_layer_count = len(dense_layer_sizes)\n        for layer_index in range(1, dense_layer_count):\n            dense_layers.append((\"critique_dense_{}\".format(layer_index), nn.Linear(self.dense_layer_sizes[layer_index-1], self.dense_layer_sizes[layer_index])))\n            dense_layers.append( ( \"critique_dense_lrelu_{}\".format( layer_index ), nn.LeakyReLU(0.2) ) )\n\n        dense_layers.append( (\"encoder_dense_{}\".format( len(self.dense_layer_sizes) ), nn.Linear( self.dense_layer_sizes[-1], 1) ) )\n        dense_layers.append( ( \"encoder_dense_sigmoid_{}\".format( len(self.dense_layer_sizes) ), nn.Sigmoid() )  )\n\n        self.dense_layers = nn.Sequential(OrderedDict(dense_layers))\n        \n    def forward(self, x):\n        \n        #print(\"x1 s\", x.shape)\n        \n        x = self.conv_layers(x)\n        \n        #print(\"x2 s\", x.shape)\n        \n        x = self.flatten(x)\n        \n        #print(\"x3 s\", x.shape)\n        \n        yhat = self.dense_layers(x)\n        \n        #print(\"yhat s\", yhat.shape)\n        \n        return yhat<\/code><\/pre>\n\n\n\n
The constructor of the Critique model class takes the following arguments: the size of a square image, the number of image channels, a sequence of channel counts for the convolution layers, the size of the convolution kernels, and a sequence of unit counts for the conventional neural network layers (with the unit count of 1 for the last layer missing, since this layer is added anyway). The model class can be instantiated as follows:<\/p>\n\n\n\n
image_size = 128\nimage_channels = 3\ncrit_conv_channel_counts = [ 8, 32, 128, 512 ]\ncrit_conv_kernel_size = 5\ncrit_dense_layer_sizes = [ 128 ]\n\ncritique = Critique(image_size, image_channels, crit_conv_channel_counts, crit_conv_kernel_size, crit_dense_layer_sizes).to(device)\n<\/code><\/pre>\n\n\n\n
The shapes of the input and output tensors for this model are as follows:<\/p>\n\n\n\n
  • input tensor: batch_size  x  image_channels x image_size  x image_size<\/li>
  • output tensor: batch_size x 1<\/li><\/ul>\n\n\n\n
    Create Generator Model<\/h3>\n\n\n\n
    For generating synthetic images from a one dimensional vector of random values, the Generator model first passes the noise vector through several conventional neural network layers, then un-flattens the output of the last ANN layer into a two dimensional feature map. This feature map is then passed through several deconvolution layers with the last one outputting the synthetic image. <\/p>\n\n\n\n
    The ANN part of the model successively increases the dimension of the noise vector. Each ANN layer with the exception of the last one is followed by a ReLU activation function. <\/p>\n\n\n\n
    The CNN part of the model successively increases the size of the feature maps while decreasing the number of channels. Each deconvolution layer with the exception of the last one is preceded by a  batch normalisation and followed by a leaky ReLU activation function. The last deconvolution layer is preceded by a  batch normalisation and followed by a Sigmoid activation function.<\/p>\n\n\n\n
    The class definition of the Generator model is as follows:<\/p>\n\n\n\n
    class Generator(nn.Module):\n    \n    def __init__(self, latent_dim, image_size, image_channels, conv_channel_counts, conv_kernel_size, dense_layer_sizes):\n        super().__init__()\n        \n        self.latent_dim = latent_dim\n        self.image_size = image_size\n        self.image_channels = image_channels\n        self.conv_channel_counts = conv_channel_counts\n        self.conv_kernel_size = conv_kernel_size\n        self.dense_layer_sizes = dense_layer_sizes\n        \n        # create dense layers\n        dense_layers = []\n                \n        dense_layers.append((\"generator_dense_0\", nn.Linear(latent_dim, self.dense_layer_sizes[0])))\n        dense_layers.append((\"generator_relu_0\", nn.ReLU()))\n        \n        dense_layer_count = len(dense_layer_sizes)\n        for layer_index in range(1, dense_layer_count):\n            dense_layers.append((\"generator_dense_{}\".format(layer_index), nn.Linear(self.dense_layer_sizes[layer_index-1], self.dense_layer_sizes[layer_index])))\n            dense_layers.append( ( \"generator_dense_relu_{}\".format(layer_index), nn.ReLU() ) )\n            \n        last_conv_layer_size = int(image_size \/\/ np.power(2, len(conv_channel_counts)))\n        preflattened_size = [conv_channel_counts[0], last_conv_layer_size, last_conv_layer_size]\n        dense_layer_output_size = conv_channel_counts[0] * last_conv_layer_size * last_conv_layer_size\n\n        print(\"preflattened_size \", preflattened_size)\n    \n        dense_layers.append( ( \"generator_dense_{}\".format(len(self.dense_layer_sizes) ), nn.Linear( self.dense_layer_sizes[-1], dense_layer_output_size) ) )\n        \n        self.dense_layers = nn.Sequential(OrderedDict(dense_layers))\n        \n        self.unflatten = nn.Unflatten(dim=1, unflattened_size=preflattened_size)\n        \n        # create convolutional layers\n        conv_layers = []\n        \n        stride = (self.conv_kernel_size - 1) \/\/ 2\n        padding = stride\n        output_padding = 1\n        \n        conv_layer_count = len(conv_channel_counts)\n        for layer_index in range(1, conv_layer_count):\n            conv_layers.append((\"generator_bnorm_{}\".format(layer_index), nn.BatchNorm2d(conv_channel_counts[layer_index-1])))\n            conv_layers.append((\"generator_conv_{}\".format(layer_index), nn.ConvTranspose2d(conv_channel_counts[layer_index-1], conv_channel_counts[layer_index], self.conv_kernel_size, stride=stride, padding=padding, output_padding=output_padding)))\n            conv_layers.append((\"generator_lrelu_{}\".format(layer_index), nn.LeakyReLU(0.2)))\n            \n        conv_layers.append((\"generator_bnorm_{}\".format(conv_layer_count), nn.BatchNorm2d(conv_channel_counts[-1])))\n        conv_layers.append((\"generator_conv_{}\".format(conv_layer_count), nn.ConvTranspose2d(conv_channel_counts[-1], self.image_channels, self.conv_kernel_size, stride=stride, padding=padding, output_padding=output_padding)))\n        conv_layers.append( ( \"generator_sigmoid_{}\".format( conv_layer_count), nn.Sigmoid() ) )\n        \n        self.conv_layers = nn.Sequential(OrderedDict(conv_layers))\n\n    def forward(self, x):\n        \n        #print(\"x1 s \", x.shape)\n        \n        x = self.dense_layers(x)\n        \n        #print(\"x2 s \", x.shape)\n        \n        x = self.unflatten(x)\n        \n        #print(\"x3 s \", x.shape)\n\n        yhat = self.conv_layers(x)\n        \n        #print(\"yhat s \", yhat.shape)\n\n        return yhat<\/code><\/pre>\n\n\n\n
    The constructor of the Generator model class takes the following arguments: the latent dimension of the image encoding, the size of a square image, the number of image channels, a sequence of channel counts for the deconvolution layers (with the channel count of 3 for the last layer missing, since this layer is added anyway), the size of the deconvolution kernels, and a sequence of unit counts for the conventional neural network layers. The model class can be instantiated as follows:<\/p>\n\n\n\n
    latent_dim = 64\nimage_size = 128\ngen_conv_channel_counts = [ 512, 128, 32, 8 ]\ngen_conv_kernel_size = 5\ngen_dense_layer_sizes = [ 128 ]\n\ngenerator = Generator(latent_dim, image_size, image_channels, gen_conv_channel_counts, gen_conv_kernel_size, gen_dense_layer_sizes).to(device)\n<\/code><\/pre>\n\n\n\n
    The shapes of the input and output tensors for this model are as follows:<\/p>\n\n\n\n
    • input tensor: batch_size  x  latent_dim <\/li>
    • output tensor: batch_size  x  image_channels x image_size  x image_size<\/li><\/ul>\n\n\n\n
      Optimisers and Loss Functions<\/h2>\n\n\n\n
      To update the weights of the two models during training, two individual Adam optimisers are used. In this example, both optimisers use the same learning rate. These optimisers are instantiated as follows:<\/p>\n\n\n\n
      gen_learning_rate = 1e-4\ncrit_learning_rate = 1e-4\n\ncritique_optimizer = torch.optim.Adam(critique.parameters(), lr=crit_learning_rate)\ngenerator_optimizer = torch.optim.Adam(generator.parameters(), lr=gen_learning_rate)<\/code><\/pre>\n\n\n\n
      Two different loss functions are used, one for the Critique model and one for the Generator model. The loss functions internally use binary cross-entropy loss to quantify the classification error of the Critique model. <\/p>\n\n\n\n
      bce_loss = nn.BCELoss()<\/code><\/pre>\n\n\n\n
      The loss function for the Critique model is named “crit_loss”. It calculates two individual losses with are then scaled and summed to obtain a single loss. The first loss is based on the difference between the Critique’s classification of “real” images (the ones that are in the dataset) and a vector containing values of 1. The second loss is based on the difference between the Critique’s classification of “fake” images (the ones output by the Generator model) and a vector containing values of 0. If the Critique makes no mistakes, then it would produce for the classification of “real”  images a vector containing values of 1 and for the classification of “fake” images a vector containing values of 0. The binary cross entropy loss between these vectors drive the training of the Critique.<\/p>\n\n\n\n
      The definition of the “crit_loss” is as follows:<\/p>\n\n\n\n
      \n# crictique loss function\ndef crit_loss(crit_real_output, crit_fake_output):\n    _real_loss = bce_loss(crit_real_output, torch.ones_like(crit_real_output).to(device))\n    _fake_loss = bce_loss(crit_fake_output, torch.zeros_like(crit_fake_output).to(device))\n\n    _loss = (_real_loss + _fake_loss) * 0.5\n    return _loss<\/code><\/pre>\n\n\n\n
      The loss function for the Generator model is named “gen_loss”. It calculates a single loss. This loss is based on the difference between the Critique’s classification of “fake” images and a vector containing values of 1. The Generator is successful if the synthetic images it generates are classified as real by the Critique.  In this case, the Critique  produces a vector containing values of 1. The binary cross entropy loss between these vectors drive the training of the Generator.<\/p>\n\n\n\n
      The definition of the “gen_loss” is as follows:<\/p>\n\n\n\n
      def gen_loss(crit_fake_output):\n    _loss = bce_loss(crit_fake_output, torch.ones_like(crit_fake_output).to(device))\n    return _loss<\/code><\/pre>\n\n\n\n
      Training and Testing Functions<\/h2>\n\n\n\n
      A total of four different functions are defined for conducting training and testing steps: a training and a testing step function for the Critique model and a training and testing step function for the Generator model. The functions for the Critique model are named “crit_train_step” and “crit_test_step” and take as input two tensors, one representing a batch of real images, the other a batch of noise vectors. The functions for the Generator model are named “gen_train_step” and “gen_test_step” and take as input one tensor representing a batch of noise vectors. All these functions compute and return loss values. The functions used for training also calculate the gradients of the loss functions and update the trainable parameters of the Critique and Generator model, respectively. The functions used for testing suppresse gradient calculation and leave the trainable model parameters unchanged.<\/p>\n\n\n\n
      def crit_train_step(real_images, random_encodings):\n    \n    critique_optimizer.zero_grad()\n    \n    with torch.no_grad():\n        fake_output = generator(random_encodings)\n    real_output = real_images\n        \n    crit_real_output =  critique(real_output)\n    crit_fake_output =  critique(fake_output)   \n    \n    _crit_loss = crit_loss(crit_real_output, crit_fake_output)\n        \n    _crit_loss.backward()\n    critique_optimizer.step()\n    \n    return _crit_loss\n\ndef crit_test_step(real_images, random_encodings):\n    with torch.no_grad():\n        fake_output = generator(random_encodings)\n        real_output = real_images\n        \n        crit_real_output =  critique(real_output)\n        crit_fake_output =  critique(fake_output)   \n    \n        _crit_loss = crit_loss(crit_real_output, crit_fake_output)\n\n    return _crit_loss\n\ndef gen_train_step(random_encodings):\n    \n    generator_optimizer.zero_grad()\n    \n    generated_images = generator(random_encodings)\n    \n    crit_fake_output = critique(generated_images)\n    \n    _gen_loss = gen_loss(crit_fake_output)\n    \n    _gen_loss.backward()\n    generator_optimizer.step()\n \n    return _gen_loss\n\ndef gen_test_step(random_encodings):\n    with torch.no_grad():\n        generated_images = generator(random_encodings)\n    \n        crit_fake_output = critique(generated_images)\n    \n        _gen_loss = gen_loss(crit_fake_output)\n    \n    return _gen_loss<\/code><\/pre>\n\n\n\n
      The function named \u201ctrain\u201d performs the actual training of the two models by calling the train and test step functions repeatedly.  This function takes as arguments the train and test Dataloaders and the number of epochs. It then runs through an outer loop and two inner loops. The outer loop iterates over all epochs. The first inner loop iterates over all the batches provided by the train Dataloader, The second inner loop iterates over all the batches provided by the test Dataloader.  In each of these inner loops, the loss returned by the loss functions are added to a dictionary. This dictionary contains the history of the training process. The training function is defined as follows:<\/p>\n\n\n\n
      def train(train_dataloader, test_dataloader, epochs):\n    \n    loss_history = {}\n    loss_history[\"gen train\"] = []\n    loss_history[\"gen test\"] = []\n    loss_history[\"crit train\"] = []\n    loss_history[\"crit test\"] = []\n    \n    for epoch in range(epochs):\n\n        start = time.time()\n        \n        crit_train_loss_per_epoch = []\n        gen_train_loss_per_epoch = []\n        \n        for train_batch, _ in train_dataloader:\n            train_batch = train_batch.to(device)\n            \n            random_encodings = torch.randn((train_batch.shape[0], latent_dim)).to(device)\n\n            # start with critique training\n            _crit_train_loss = crit_train_step(train_batch, random_encodings)\n            \n            _crit_train_loss = _crit_train_loss.detach().cpu().numpy()\n\n            crit_train_loss_per_epoch.append(_crit_train_loss)\n            \n            # now train the generator\n            for iter in range(2):\n                _gen_loss = gen_train_step(random_encodings)\n            \n                _gen_loss = _gen_loss.detach().cpu().numpy()\n            \n                gen_train_loss_per_epoch.append(_gen_loss)\n        \n        crit_train_loss_per_epoch = np.mean(np.array(crit_train_loss_per_epoch))\n        gen_train_loss_per_epoch = np.mean(np.array(gen_train_loss_per_epoch))\n\n        crit_test_loss_per_epoch = []\n        gen_test_loss_per_epoch = []\n        \n        for test_batch, _ in test_dataloader:\n            test_batch = test_batch.to(device)\n            \n            random_encodings = torch.randn((train_batch.shape[0], latent_dim)).to(device)\n            \n            # start with critique testing\n            _crit_test_loss = crit_test_step(train_batch, random_encodings)\n            \n            _crit_test_loss = _crit_test_loss.detach().cpu().numpy()\n\n            crit_test_loss_per_epoch.append(_crit_test_loss)\n            \n            # now test the generator\n            _gen_loss = gen_test_step(random_encodings)\n            \n            _gen_loss = _gen_loss.detach().cpu().numpy()\n            \n            gen_test_loss_per_epoch.append(_gen_loss)\n\n        crit_test_loss_per_epoch = np.mean(np.array(crit_test_loss_per_epoch))\n        gen_test_loss_per_epoch = np.mean(np.array(gen_test_loss_per_epoch))\n        \n        if epoch % weight_save_interval == 0 and save_weights == True:\n            torch.save(critique.state_dict(), \"results\/weights\/critique_weights_epoch_{}\".format(epoch))\n            torch.save(generator.state_dict(), \"results\/weights\/generator_weights_epoch_{}\".format(epoch))\n        \n        plot_gan_outputs(generator, epoch, n=5)\n        \n\n        loss_history[\"gen train\"].append(gen_train_loss_per_epoch)\n        loss_history[\"gen test\"].append(gen_test_loss_per_epoch)\n        loss_history[\"crit train\"].append(crit_train_loss_per_epoch)\n        loss_history[\"crit test\"].append(crit_test_loss_per_epoch)\n\n        print ('epoch {} : gen train: {:01.4f} gen test: {:01.4f} crit train {:01.4f} crit test {:01.4f} time {:01.2f}'.format(epoch + 1, gen_train_loss_per_epoch, gen_test_loss_per_epoch, crit_train_loss_per_epoch, crit_test_loss_per_epoch, time.time()-start))\n    \n    return loss_history<\/code><\/pre>\n\n\n\n
      To visually verify the progress of the training, the training function calls after each epoch a function named “plot_gan_outputs”. This function draws an image in which several synthetic images generated by the Generator are placed in a row. This function takes as arguments the Generator model, the current epoch, and the number of images that the Generator should generate. This function is defined as follows.<\/p>\n\n\n\n
      def plot_gan_outputs(generator, epoch, n=5):\n    \n    generator.eval()\n    \n    plt.figure(figsize=(10,4.5))\n    for i in range(n):\n      ax = plt.subplot(1,n,i+1)\n      \n      generator.eval()\n      with torch.no_grad():\n         random_encoding = torch.randn((1, latent_dim)).to(device)\n         gen_img  = generator(random_encoding)    \n      generator.train()\n      \n      gen_img = gen_img.cpu().squeeze().numpy()\n      gen_img = np.clip(gen_img, 0.0, 1.0)\n      gen_img = np.moveaxis(gen_img, 0, 2)\n      \n      plt.imshow(gen_img)\n      ax.get_xaxis().set_visible(False)\n      ax.get_yaxis().set_visible(False)  \n      if i == 0:\n          ax.set_title(\"Epoch {}: Generated Images\".format(epoch))\n    plt.show()\n    \n    generator.train()\n<\/code><\/pre>\n\n\n\n
      The \u201ctrain\u201d function can be called as follows:<\/p>\n\n\n\n
      epochs = 1000\n\nloss_history = train(train_dataloader, test_dataloader, epochs)<\/code><\/pre>\n","protected":false},"excerpt":{"rendered":"
      Summary The following tutorial introduces the use of a simple generative adversarial network for generating synthetic images. This model can be trained on a collection of images. The model uses a combination of a conventional neural network (ANN) and a convolutional neural network (CNN) to create images. This tutorial forms part of a series of […]<\/p>\n","protected":false},"author":2154,"featured_media":0,"parent":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"_coblocks_attr":"","_coblocks_dimensions":"","_coblocks_responsive_height":"","_coblocks_accordion_ie_support":"","footnotes":""},"class_list":["post-1821","page","type-page","status-publish","hentry"],"_links":{"self":[{"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/pages\/1821","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/pages"}],"about":[{"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/types\/page"}],"author":[{"embeddable":true,"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/users\/2154"}],"replies":[{"embeddable":true,"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/comments?post=1821"}],"version-history":[{"count":130,"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/pages\/1821\/revisions"}],"predecessor-version":[{"id":3368,"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/pages\/1821\/revisions\/3368"}],"wp:attachment":[{"href":"https:\/\/wp.coventry.domains\/e2edu\/wp-json\/wp\/v2\/media?parent=1821"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}