DEEP LEARNING BASED STRESS PREDICTION FOR BOTTOM-UP

STEREO-LITHOGRAPHY (SLA) 3D PRINTING PROCESS

by

Aditya Pramod Khadilkar

September 2018

A thesis submitted to the faculty of the Graduate School of the University at Buffalo,

State University of New York in partial fulfillment of the requirements for the degree

of Master of Science

Department of Mechanical and Aerospace Engineering

Table of Contents

List of Figures………………………………………………………………………...iii
List of Tables…………………………………………………………………………iii
Abstract……………………………………………………………………………….iv
1. Introduction………………………………………………………………………... 1
2. Literature Review………………………………………………………………….. 4
2.1. Separation process and FEA models…………………………………….. 4
2.2 FEA and NN based prediction models…………………………………… 5
2.3 Convolutional Neural Networks………………………………………….. 6
3. Overview………………………………………………………………………….. 8
4. Database Creation………………………………………………………………… 10
4.1 3D CAD Database……………………………………………………….. 10
4.2 Input Data Creation……………………………………………………… 12
4.3 Output Data Creation……………………………………………………. 14
4.4 Data Augmentation……………………………………………………… 18
5. Convolutional Neural Network Architectures……………………………………. 20
5.1 Convolutional Neural Network………………………………………….. 20
5.2 Two-Stream Convolutional Neural Network……………………………. 22
6. Results and Discussion……………………………………………………………. 24
6.1 Stress Prediction using CNN-1………………………………………….. 24
6.2 Effect on Non-uniform 3D parts………………………………………… 25
6.3 Stress Prediction using 2-stream CNN………………………………….. 26
6.4 Computational Time…………………………………………………….. 27
7. Conclusion and Future Work……………………………………………………… 29
References…………………………………………………………………………... 31

ii
List of Figures

Figure 1. Top-Down printing process……………………………………………….. 2

Figure 2. Bottom-Up printing process……………………………………………..... 2
Figure 3. Overall pipeline of thesis work……………………………………………. 8
Figure 4. 12 sample CAD Models generated using Inventor VBA……………….... 12
Figure 5. STEP files generated for each slice using Inventor VBA…………………12
Figure 6. Input data generation process…………………………………………….. 13
Figure 7. Setup for FEA simulation in ABAQUS………………………………….. 14
Figure 8. Stress distribution on cured layer………………………………………… 15
Figure 9. Stress-Time behavior during separation process…………………………. 15
Figure.10 CAD models selected for peak stress analysis…………………………… 16
Figure 11. (a-j): Stress-Time curve for FEA simulation of 5 parts at each layer ……17
Figure 12. Stress extraction process at grid locations ……………………………… 18
Figure 13. CNN-1 network with 3 convolutional layers, 1 max-pooling layer and 2
fully connected layer…………………………………………………………………20
Figure 14. 2-stream CNN network where (+) denotes concatenation of 2 feature
vectors………………………………………………………………………………..22
Figure 15. Plot for convergence of L2 loss and prediction error of CNN-1………… 24

Figure 16. Comparison in prediction error between CNN and NN……………….....25

Figure 17. Effect of non-uniform 3D parts on maximum stress generated on the cured
layer…………………………………………………………………………………. 25

Figure 18. Plot for convergence of L2 loss and prediction error of 2-stream CNN…. 26

Figure 19. Comparison of prediction error between CNN-1 and 2-Stream CNN……27

Figure 20. Computational time for FEA simulations for 5 different parts at each
layer…………………………………………………………………………………. 28

List of Tables

Table 1. Different cross-sections and their dimensions used for generating dataset... 11

iii
Abstract

Additive Manufacturing (AM) is a fabrication process which creates a 3D part in a

layer-by-layer fashion. AM allows fabrication of complex geometric parts that are

difficult to fabricate using a traditional subtractive manufacturing process. Stereo-

lithography (SLA) printing is an AM technique which prints the 3D part from liquid

resin based on the principle of photo-polymerization. Part deformation and failure

during the separation process are the key bottlenecks in printing high-quality parts using

bottom-up SLA printing. Cohesive Zone Models have been successfully used to model

the separation process in bottom-up SLA printing process. However, the Finite Element

(FE) simulation of the separation process is prohibitively computationally expensive

and thus cannot be used for online monitoring of the SLA printing process. This thesis

presents a Deep Learning (DL) based framework to predict the stress distribution on

the cured layer of the printed part in real time. The framework consists of (1) a new 3D

model database that captures a variety of geometric features that can be found in real

3D parts and (2) FE simulation on the 3D models present in the database that is used to

create inputs and corresponding labels (outputs) to train the DL network. Two different

types of DL networks were trained network to predict the stress using the test dataset.

Comparison between two different DL networks shows the validity of the proposed

framework for prediction of stress in complex 3D parts. Results further show that

computational time is drastically reduced by this framework in comparison with FE

simulations.

iv
1. Introduction

Additive manufacturing (AM), also known as 3D printing, is a technique of producing

finished components using a layer by layer deposition of material unlike traditional

material removal manufacturing processes, known as subtractive manufacturing. AM

techniques offer more flexibility to designers as complex and small parts, which cannot

be manufactured using subtractive manufacturing, can be fabricated using AM

techniques. However, slow printing speeds, low surface finish, inferior mechanical

properties and need for post-processing operations still pose a challenge to its

widespread use in the industry.

Stereo-lithography Apparatus (SLA) was the first 3D printer developed in 1980’s and

patented by Charles Hull. Stereo-lithography (SL) is a Solid Freeform Fabrication

(SFF) technique with a superior resolution of around 20µm [2]. SLA uses the principle

of photo-polymerization to build the 3D computer aided design (CAD) model layer

upon layer with high resolution and superior surface finish. During the printing process,

a layer of liquid is solidified using a laser beam or a light source. Based on the light

projection techniques, SLA printing is classified as (i) free-surface (top-down) printing

and (ii) constrained-surface (bottom-up) printing. Among these, constrained-surface

image-projection-based SLA printing (bottom-up) will be the focus of this paper.

Fig. 1 and 2 show schematic of top-down and bottom-up 3D printers. In a top-down

printer, the platform is lowered into the vat as each layer is printed. As opposed to this,

the platform in a bottom-up printer is raised after printing each layer. Bottom-up

printing has many advantages over top-down printing like less production time, less

material waste, faster curing time and higher resolution [4]. However, bottom-up

printing has its own disadvantages as well. When the platform is raised after curing, the

1
layer adheres to the vat surface that creates separation forces at the layer-vat interface

[7]. These forces could lead to deformation and consequently failure of the printed

parts. A coating of Teflon or Polydimethylsiloxane (PDMS) silicone film applied to the

vat reduces the separation forces. Fig. 2 (a-d) shows the separation process with the

PDMS film on vat. Although the silicon film reduces the separation forces, it was found

that the forces were still large enough to cause part deformation and failure [10].

Figure 1. Top-Down printing process

Figure 2. Bottom-Up printing process

2
Many methods have been developed to model the separation behavior at layer-vat

interface [3, 7]. Finite Element (FE) simulations using these models yield stresses

induced due to separation forces at given velocity. These simulations tend to be

computationally expensive and thus cannot be used for online monitoring of the

printing process [8]. The objective of this work is to develop a deep learning (DL)

framework using convolutional neural networks (CNN) to calculate the stress induced

in any layer of a given 3D CAD model in real time to help with online monitoring of

the bottom-up SLA printing process. To train the network, a dataset was created using

Autodesk Inventor API and ABAQUS python script was used to carry out FE

simulations on the generated dataset.

This thesis makes following two key contributions:

(1) Developed a CNN based DL framework to efficiently compute stress on the

cured layer of non-uniform 3D parts.

(2) Created a large-scale dataset with FE simulations to train CNN based DL

network.

3
2. Literature review

2.1. Separation process and FEA models

Many researchers have attempted to reduce or eliminate the attachment force. As

discussed earlier, the application of Teflon or silicon film like Polydimethylsiloxane

(PDMS) silicone helps in separation of the cured layer from the vat surface [6, 9].

However, application of such film do not lead to significant force reductions [10].

EnvisionTec Inc. developed a peeling mechanism where one side of the platform is

raised so as to peel the part off vat surface [11]. Although the developed mechanism

reduces the force required to separate the cured layer, it works efficiently only for parts

with geometries that lead to high cross sectional area [4]. A two-channel system

introduced by Zhou et. al. [5] enabled significant reduction in the separation force. They

divided the vat into two channels where one half was coated with PDMS film and the

other half is not coated. The part is printed on PDMS film and the moved horizontally

first till it reaches the half which is uncoated. Then, using vertical and horizontal

movements, the part is aligned for curing of the next layer. This helps in reducing the

separation forces as only shearing force acts on the cured layer during first horizontal

motion. However, extra horizontal motion increases the fabrication time and shearing

stresses deform part in the horizontal direction. Although this system reduces the

separation force, the separation forces are still large enough to cause deformation and

part failure. This has motivated many researchers to create a predictive model that can

be used for online monitoring of the printing process. Huang [6] and Zhou et. al. [5]

have shown that separation force primarily depends on following four factors: (i)

Exposure time (ii) Image area (iii) Image Geometry and (iv) Pull-up velocity. Liravi et.

al. [3] and Hang Ye et. al. [7] proposed predictive models based on Cohesive Zone

Models (CZM) to predict separation forces during pull-up forces. In their work, they

4
predicted the stresses at the cured layer for a given pull-up velocity. If the stresses are

above a threshold value, pull-up velocity would be reduced to prevent part deformation.

However, FEA simulations using these models are computationally expensive and

hence reduce the printing time drastically if employed while printing a part. This creates

a need for faster stress prediction that can help in providing in-situ feedback to the

system. Jun et. al. [8] developed such framework using FEA and NN which drastically

reduced computational time. This work follows on their work using convolutional

neural networks (CNN).

2.2. FEA and NN based prediction models

FEA and NN based prediction models have been successfully implemented previously.

Cao et. al. [12] used FEA and a back-propagation (BP) NN to predict rolling force in

hot rolling of electrical steel. Reddy et. al. [14] used NN to predict temperature

distributions online in electron beam-welded plates. They collected the data required

for training using FEA analysis. Shahani et. al. [13] used an artificial neural network

(ANN) to predict slab behavior in the hot rolling process. They used different

parameters such as Friction coefficient, temperature, rolling speed, and % reduction as

input to the network to predict various behaviors such as temperature distribution, strain

rate, deformation and rolling force. FEA simulation was carried out on 25 training

samples to create a training dataset. Hu et. al. [15] created a prediction model for

predicting failure pressure of composite cylinders for hydrogen storage. A prediction

model for spring back in wipe-bending process was developed using BP NN by [16].

Training data was obtained using FEA simulations on several samples. Umbrello et. al.

[17] proposed a hybrid FE-ANN model for predicting optimal cutting conditions and

residual stresses in bearing steel. Arndt et. al. [18] used NN to approximate the

5
simulation results based on FEA. An optimization strategy was used to integrate the

approximation results. Hu et. al. [19] developed multi-scale hierarchical model using

FEA and NN to link mesoscopic and microscopic scales to simulate the bone

remodeling process. Liang et. al. [20] developed a deep learning (DL) model to predict

stress distribution in the aorta. They used Principle Component Analysis (PCA) for

shape encoding of the aorta and multi-layer neural network was used to calculate stress

distribution. Jun et. al. [8] presented first work in using FEA and NN based prediction

model for bottom-up SLA printing process. N-fold symmetric shapes were used for

training the single layer neural network. 2D shape descriptor was used for shape

encoding of input shapes. Output used for training was FEA simulation results using

ABAQUS. CZM based simulation model was implemented for FEA simulations. The

dataset size was limited to 128 during the training phase. However, it did not

encapsulate the wide array of cross-sections in real-world 3D printed parts. Also, the

cross-sections of the part were constant and it did not predict stress information at each

layer. Although this work is first to use FEA and NN for stress prediction in bottom-up

SLA printing, more work is required to improve the accuracy of the prediction model

for complex 3D shapes. This thesis investigates the use of a convolutional neural

network model to predict stress information on each layer.

2.3. Convolutional Neural Networks

Convolutional Neural Network (CNN) was first developed by LeCun [21] in 1998,

which efficiently classified MNIST dataset using 7 layers of convolutions and

subsampling. However, due to CPU and GPU restrictions, it was not used widely for

further ten years. Recently, due to the fast computing capabilities of modern CPUs and

GPUs, CNNs have become ubiquitous in image classification and object recognition

tasks. Alex et. al. [22] proposed AlexNet, a CNN architecture which classified

6
ImageNet dataset with 17.0% top-5 error. They employed ReLU non-linearity

combined with local response normalization and Dropout to achieve these results.

VGGNet [25] and GoogLeNet [23] were recently introduced which were deeper

networks with more convolutional layers. GoogLeNet is a 22-layer deep network. All

these networks work towards the task of classification, but stress calculation is a

regression problem and not a classification problem. DeepPose [24] was developed in

2014 which was used for human pose estimation. DeepPose treated the pose estimation

problem as regression problem, where the output pose is represented in a normalized

pose vector function of the input image. In DeepPose, an architecture similar to AlexNet

architecture was used with a different loss function. L2 loss was computed and used to

optimize the parameters using back-propagation. A cascade of pose regressors was used

to refine the prediction. This thesis utilizes the CNN for regression to predict stress on

each layer of the 3D printed part. To the best of our knowledge, CNNs have not been

employed previously to determine stresses. This thesis investigates the use of CNN in

stress prediction problems.

7
3. Overview

As discussed in section 2, stress distribution on the cured layer depends on various

different parameters. These parameters need to be considered while creating a

predictive model. Stress prediction using neural networks has been attempted

previously by [8], but their method has its limitations. In [8], the cross section area is

kept constant for the entire dataset, which poorly imitates the real-world applications.

Secondly, the predictive model was based on a single layer of a part with a uniform

cross-section. Thirdly, the neural network was trained on a database of 128 parts which

is not sufficiently large for effectively training a neural network. This work tries to

overcome these limitations by implementing a CNN using a large dataset with varying

cross-section at each layer. Fig 3. Shows the overall pipeline of this work.

Figure 3. The overall pipeline of thesis work

Dataset can either be real-world data or it can be synthetically produced. We generated

data for training our model synthetically using Autodesk Inventor VBA and ABAQUS.

Dataset was generated so that it can encompass cross-sections of different shapes and

8
sizes at different layers. Details about dataset generation are given in section 4. A CNN

was trained using the generated data to predict the stress distribution on each layer. The

CNN consists of eight layers- the input layer, three convolutional layers, one pooling

layer, two fully connected layers and the final prediction layer. The architecture of CNN

is explained in section 5. The results of training the CNN on the curated database led

us to investigate another factor affecting the stress prediction which is further discussed

in section 6. We developed a two-stream CNN to improve the predictive model for

stress distribution. The two-stream CNN architecture consists of two parallel CNNs

concatenated to predict stress on the cured layer. This architecture is further explained

in section 5.

Tensorflow library was used to implement and train the CNN network. The entire

database was split in ratio 80:10:10 for training, validation, and testing.

9
4. Database Creation

4.1 3D CAD database

SLA printing allows for users to print a vast array of models that are not possible to

fabricate using traditional manufacturing techniques. Hence, a layer can have any shape

depending on the 3D model and dimension bounds depending on the printing volume

of the 3D printer. In order to get a better generalization, an ideal training dataset should

encompass a wide variety of shapes and sizes. We created a dataset of 3D CAD models

with varying cross-section in an automated fashion. Each 3D model was created using

loft on 3 different sketches on 3 planes at 5mm apart from each other. These 3 sketches

were selected using permutations from a repository of 12 sketches. The 12 sketches

were made from 6 shapes each having 2 different dimensions. Following six shapes

were selected: Ellipse, Triangle, Rectangle, Star, Plus and Pentagon. Table 1 shows

different shapes and their dimensions. Depending on the feasibility of loft, 1670 CAD

models were created using combinations of 3 sketches from these 12 sketches.

Shape Name Dimensions (mm)

Rectangle a=[14,16]

Ellipse a=[12,14]

10
 a

Pentagon a=[15,17.5]

Triangle a=[20, 25]

a
Star a=[15, 17.5]

Cross a=[5,6]

Table 1. Different cross-sections and their dimensions used for generating the dataset

The direction of the loft was y-axis and was kept constant for the whole dataset. Using

loft on different shapes helped us create cross-sections of varying size and shape at each

layer. A script was written to automatically generate all the CAD files using Inventor

VBA. Sample 3D models created using the automated script is shown in Fig. 4.

11
 Figure 4. 12 sample CAD Models generated using Inventor VBA

After creating the CAD files, each file was sliced in 10 layers using Inventor VBA

code and saved as a separate CAD file. While slicing the CAD file, previous layers

were retained in the model such that the model replicates layer-by-layer printing

process. Each model was saved as a different STEP file for further FEA analysis. Thus,

16700 CAD files were created in total which made up the initial database. Fig. 5 shows

a part sliced into 10 layers and each layer saved as STEP file for FE simulation.

Figure 5. STEP files generated for each slice using Inventor VBA

4.2 Input data creation

A major step of training neural networks is dataset creation. After generating the CAD

files, we generate the input and the labels for training the network. CNN takes an image

as input. Binary images of the cross-section of all the layers in a CAD model with 128

x 128 resolution were created. Fig. 6 shows the process of creating an image dataset.

12
Once a 3D model is sliced in slices, the cross-section of the printed slice is saved in

STL mesh format. A grid of 128x128 in a bounding box of size x = [-20, 20] mm and

y = [-20, 20] mm is superimposed on the STL mesh file. The points in the grid which

lie on the surface or inside of the mesh are detected and assigned a value of 1 and all

other grid points which do not lie on the surface or fall outside the boundary of the mesh

are assigned a value of 0. This results in a binary image of the cross-section of the

printed slice. A MATLAB code was used for creating the binary images from STL files.

Figure 6. Input data generation process

The image is a representation of a cross-section of the printed part at the part-vat

interface. These images are used for training the proposed CNN architecture in section

5.1. For the 2-stream architecture explained in section 5.2, a second input to the model

is the stress distribution from the previous stage. Stress distribution input is a 10000 bit

13
vector storing the stress prediction values at different locations of the previous layer.

During training, FEA results from the previous layer were transformed to an array of

size (100x100) using method explained in the next section. This becomes input for the

second stream of CNN.

4.3 Output data creation

The neural network is trained to predict the stress distribution on the slice layer area

that is in contact with PDMS film. The stress distribution is calculated using FEA

simulation. The FEA simulation was enabled through the use python script that

repeatedly utilized the services of ABAQUS software. The details of the used FEA

model is covered in an earlier publication [4]. Fig. 7 shows the setup for FEA

simulation. Constant velocity is applied at the top surface of the 3D part. Cohesive Zone

Model (CZM) is used to model the behavior at the layer-vat interface.

Part

PDMS
Film

Figure 7. Setup for FEA simulation in ABAQUS

FEA simulation was done on all 16,700 parts in ABAQUS with an average time of 1.25

minutes per simulation. Fig. 8 shows the stress distribution on the cured layer of the

part shown in Fig. 7.

14
 Figure 8. Stress distribution on the cured layer

The FE model was simulated for 8 seconds. As reported in [8], the stress behavior with

respect to time can be plotted as shown in Fig. 9

Critical Region

Figure 9. Stress-Time behavior during the separation process

It can be seen that the stress goes on increasing before reaching a peak value and then

decreases sharply to zero. This drastic reduction in the stress value denotes the

separation of the cured layer from the vat surface. Hence, the peak stress is critical to

determine whether the cured layer will deform and fail. The peak stress occurs generally

in the range of 3.5s – 4.5s [8]. We further investigate the occurrence of peak stress in

the generated dataset. We randomly selected 5 different 3D parts generated using

Inventor VBA and perform peak stress on these 5 parts at each layer. Fig. 11 (a-j) shows

stress-time curves for each layer of 5 different parts shown in Fig. 10.

15
Part 1 Part 2 Part 3 Part 4 Part 5
Figure. 10 CAD models selected for peak stress analysis

(a) Layer 1 (b) Layer 2

(c) Layer 3 (d) Layer 4

(e) Layer 5 (f) Layer 6

16
 (g) Layer 7 (h) Layer 8

(i) Layer 9 (j) Layer 10

Figure 11 (a-j): Stress-Time curve for FEA simulation of 5 parts at each layer. Part 1-
5 in legend stands for parts from fig. 9

From the Fig. 11 (a-j), it can be seen that stress increases to a maximum value (peak

value) and then decreases rapidly. The maximum occurs in the time range of [4.1s-

4.3s]. Based on the results of peak stress analysis, we create a predictive model based

on stress values at time 4.2s for each part and at each layer.

Next step in dataset curation is to create the output vector by extracting the stress data

from the simulation. A grid of size 100x100 was created with a bounding box of size x

= [-20, 20] mm and y = [-20, 20] mm. Jun et. al. [8] reported that a grid size of 100x100

gives better results than smaller grid sizes. Hence, a grid size of 100x100 was chosen

for this study. Similar to image data creation, stress at all the grid points which lie on

17
the cross-section of the cured layer was extracted and stress of 0 was assigned to all

other grid points. Fig. 12 shows an illustration of stress extraction from FEA simulation

using grid points. Green points denote the stress extraction points on the layer and red

points denote no stress has been extracted from that area and are assigned 0 value.

Figure 12. Stress extraction process at grid locations

Thus 100x100 output grids were created from all the FEA simulations and were further

reshaped into a vector of size of 10000. An output vector of size 10000 is created for

all the 16,700 CAD models.

4.4 Data Augmentation

Performance of a CNN increases with increase in the size of the dataset. Many data

augmentation techniques are used for image data such as flip, mirror, rotate, translate,

crop and noise addition. We implement rotation to our dataset to augment the size of

18
the dataset. The rotation was implemented on the dataset as FEA output is invariant to

rotation. Hence, FE analysis is not repeated for different rotations. We rotate all the

input images and corresponding output grids by 90o, 180o, and 270o. Thus, the size of

the dataset increased fourfold. The size of the overall dataset is 66,800.

19
5. Convolutional Neural Network Architectures

5.1 Convolutional Neural Network

Convolutional neural networks have proven to be successful in classification and

regression tasks. Convolutional neural networks learn spatial features from an image

with help of spanning filters over the image to achieve high accuracy in classification

and regression tasks. There has been a lot of research in this field where standard

architectures like LeNet, AlexNet, and ResNet are used for any required tasks by fine-

tuning the pre-trained model. Our architecture is based on components used in these

architectures. Fig. 13 shows the overall CNN architecture (CNN-1).

Figure 13. CNN-1 network with 3 convolutional layers, 1 max-pooling layer and 2
fully connected layer

The input to the CNN is an image with size 128x128. Following the input layer, we

have three convolutional layers. Each convolutional layer has the following parameters:

input feature maps (f0), output feature maps (f1), filter size (dxd) and stride (s). In the

Fig 13, each convolutional layer is represented by parameter (f1, d, d, s). These

parameters for three convolutional layers are (10,8,8,2), (16,8,8,2), and (16,6,6,1). In

each convolutional layer, the filters are convolved over the input feature maps. ReLU

20
activation function is used on these results to create feature maps of given layer. ReLU

function is used to introduce non-linearity in the network. The fourth layer is a max-

pooling layer used to down-sample the feature maps along the spatial dimensions from

the third convolutional layer. Parameters of the max-pooling layer are stride (s) and size

of window (dxd). The size of window in our architecture is (2x2) with a stride of 2. It

is represented as (d,d,s) in the Fig 12. Two fully connected layers follow the max-

pooling layer. The first fully connected layer gives an output vector of size 1000 i.e. it

consists of 1000 neurons. Each neuron in the fully connected layer has a ReLU

activation function similar to convolutional layers. The input to the fully connected

layer is reshaped output from the max-pooling layer. The output from a neuron in a

fully connected layer is given by the following expression:

𝑜𝑘 = 𝑔(𝑤 𝑘 ⋅ 𝑥 + 𝑏 𝑘 ) (1)

where 𝑤 𝑘 is the weight vector for kth neuron, 𝑥 is the input and 𝑏 𝑘 is bias term for kth

neuron. g is the ReLU activation function applied to the output of each neuron. (.)

denotes dot product. The second fully connected layer consists of 5000 neurons and

utilizes the same equation (equation 1). The final layer is the prediction layer that has

10000 neurons which give output based on (1) and are the stress predictions for a given

cross-section.

The loss function used for analyzing the predictions is the L2 loss function given by the

following expression:

𝑛 2
𝐿=∑ (𝑦𝑖 − ℎ(𝑥𝑖 )) (2)
𝑖=1

where n denotes the number of predictions, 𝑦𝑖 denotes the true values and ℎ(𝑥𝑖 ) denotes

the predictions. Adam optimizer was used to minimize the loss.

21
Average relative error 𝑒𝑎 over all locations is given by the following expression:

𝑛
∑𝑖=1|(𝑦𝑖 −ℎ(𝑥𝑖 ))|
𝑒𝑎 = 𝑛 × 100% (3)
∑𝑖=1|𝑦𝑖 |

5.2 Two-Stream Convolutional Neural Network

Two stream CNN architectures have been used previously for video classification tasks

[27]. Our architecture is similar to these architectures in a way that we use two parallel

networks which act as feature extractors and output from these networks is

concatenated before passing onto two fully connected layers. The two stream

architecture is shown in Fig. 14.

Figure 14. 2-stream CNN network where (+) denotes concatenation of 2 feature
vectors

In this architecture, the first stream is similar to previous architecture explained in

section 5.1. The second stream also has a similar architecture. The main difference

being the absence of max-pooling layer. The input to the second stream (input_2 in Fig.

14) is the stress distribution of the previous layer. The fully connected layer of both

streams is concatenated to form a vector of size (2000). This is followed by a fully

22
connected layer and final prediction layer. Similar to CNN-1, ReLU activation is used

for 2-stream CNN model. The L2 loss is optimized using Adam optimizer and error is

calculated using equation (3).

For 2-stream CNN, we need a prediction from the previous layer as an input. Hence,

this CNN model cannot be used to predict stress on the first layer of the part. We use

FEA simulation for the first slice or train a network as discussed in section 5.1 for

predicting the stress at the first layer of the part.

23
6. Results and Discussion

6.1 Stress prediction using CNN-1

The generated dataset of 66,800 samples was divided into three subsets- training,

validation, and testing in 80:10:10 ratio respectively. Tensorflow library was used for

the implementation of deep learning architectures. Training dataset was used to learn

around 60 million model parameters and validation set was used for further tuning the

model hyper-parameters. The final model was tested on the test dataset to measure the

performance and generalizing ability of the model. After conducting experiments with

different optimizers, Adam optimizer was chosen as it performed better than other

optimizers like SGD and momentum. The experiments were performed on a PC with

quad-core Intel i7 processor (3.40 GHz) with 16 GB RAM. Fig. 15 shows the plot for

convergence of loss and error. For the best performance for CNN-1 model, we achieved

42.45% prediction error for training and 76.90% for the validation set. Prediction error

for test dataset for this model was 78.65%. The error was considerably high for this

architecture, which led us to investigate more about the dataset. It is further discussed

in the next section. Fig. 16 shows comparison of error for CNN and NN when trained

on Jun et. al. [8] dataset.

Figure 15. Plot for convergence of L2 loss and prediction error of CNN-1. It can be
seen that the train error is high and also the model is overfitting on the training dataset

24
 Prediction Error Comparison
8.00% 7.60%

7.00%
Prediction Error (%)
6.00% 5.30%
5.00%
4.00%
3.00%
2.00%
1.00%
0.00%
NN CNN

Figure. 16 Comparison in prediction error between CNN and NN when trained on

dataset provided by Jun et. al. [8]

6.2 Effect of non-uniform 3D parts

We examined several parts with a similar cross-section at a particular layer to see the

stress distribution on that layer for a given part. We found out that different parts with

the same cross-section at a particular layer have different stress distribution on that

layer. Fig. 17 shows three different parts with the same cross-section at the 7th layer and

their corresponding stress distribution.

Figure 17. Effect of non-uniform 3D parts on maximum stress generated on the cured
layer

25
This shows that for non-uniform 3D parts, along with given layer information we need

information from the previous layers as well. This motivated us to develop a new

architecture where the stress information of the previous layer is also used for stress

prediction for a given layer.

6.3 Stress prediction using 2-stream CNN

Motivated by the findings discussed in the previous section, we developed a 2-stream

CNN architecture explained in section 5.2. Since we need stress information of the

previous layer for this architecture, this model can predict stress only from second layer

onwards. The total size of the dataset for this architecture was 60,120 which was further

split in the ratio 80:10:10 for training, validation, and testing. This architecture

improves the results drastically with prediction error for training being 6.6% and that

for validation being 13.65%. This model achieved a testing error of 15.62%. Fig. 18

shows the training and validation error and loss convergence with Adam optimizer. It

can be seen that there is a slight overfitting in the model. Overfitting can be further

reduced by using techniques like regularization and generating a larger dataset.

Figure 18. The convergence of L2 loss and prediction error of 2-stream CNN

26
 Prediction Error Comparison
100.00%
81.50%

Prediction Error (%)

80.00%

60.00%

40.00%
15.60%
20.00%

0.00%
CNN-1 2-Stream CNN

Figure. 19 Comparison of the prediction error between CNN-1 and 2-Stream CNN on
the created dataset

It was seen that 2-stream architecture performs better than CNN-1 on new larger

dataset.

6.4 Computational Time

The main motivation behind using CNN instead of FEA simulation is the computational

time required for FEA simulations which render it infeasible for real-time stress

predictions. Fig. 20 shows the computational time required for FEA simulation for 5

different parts of each layer. It can be seen that computational time increases with

increase in the number of layers.

27
 Part 1 Part 2 Part 3 Part 4 Part 5

Figure 20. Computational time for FEA simulations for 5 different parts at each layer

Computational time required by using the CNN architecture is 25.12 milliseconds

which is considerably lower than simulation time using ABAQUS.

28
7. Conclusion and future work

This thesis presents a framework for predicting stress distribution on a layer of non-

uniform 3D parts. A large labeled dataset was created to train the CNN in an automated

manner using Autodesk Inventor. This framework can be used to create even larger

dataset with more shapes. The generated dataset was sliced in 10 layers using Inventor

VBA to create STP files for FEA simulation and STL files for image data generation.

Input images were created from STL files generated using the slicing operation. FEA

simulation on 16,700 models was performed in ABAQUS to generate label data. This

dataset was used to develop and train an architecture that can predict layer-by-layer

stress distribution for a bottom-up SLA printing process. The dataset further helped in

finding out the parameters such as peak stress and dependence on previous layer

information which determine the stress distribution on a layer. The deep learning model

outperforms the simple neural network model previously used for stress prediction.

This framework can also be further used for training a larger dataset of 3D parts with

varying heights as well. This framework cannot be used to predict stress on all the layers

in a 3D part. This is due to the fact that previous layer stress information to predict

current layer stress. Using a prediction of the previous layer to predict current layer

stress induces more error due to compounding. Future work will be stress prediction on

each layer of the 3D part. Instead of using stress from the previous layer, a 3D shape

descriptor can be used for representation of previous layers. This framework can also

be used for selecting the print orientation so as to minimize the stresses induced in the

cured layer. Further analyzing the effect of the 3D model on peak stress with respect to

time can also help in improving results even more. A good direction for future research

can be incorporating more parameters like the height of slice and pull-up velocity to

29
mimic the 3D printing process more realistically and to get better control over the

process.

30
References:

[1] Tuan D. Ngo, Alireza Kashani, Gabriele Imbalzano, Kate T.Q. Nguyen, David Hui,
Additive manufacturing (3D printing): A review of materials, methods, applications
and challenges, Composites Part B: Engineering, Volume 143, 2018, 172-196

[2] Ferry P.W. Melchels, Jan Feijen, Dirk W. Grijpma, A review on stereolithography
and its applications in biomedical engineering, Biomaterials, Volume 31, Issue 24,
2010, 6121-6130

[3] Liravi F. Dynamic force analysis for bottom-up projection-based Additive

Manufacturing using finite element analysis. State University of New York at Buffalo;
2014

[4] Liravi F, Das S, Zhou C. Separation force analysis and prediction based on cohesive
element model for constrained-surface stereolithography processes. Computer Aided
Design 2015; 69:134–42

[5] Zhou C, Chen Y, Yang Z, Khoshnevis B. Digital material fabrication using

maskimage-projection-based stereolithography. Rapid Prototyp J 2013;19(3):153–65.

[6] Huang Y-M, Jiang C-P. On-line force monitoring of platform ascending rapid
prototyping system. J Mater Process Technology 2005;159(2):257–64.

[7] Ye H, Venketeswaran A, Das S, Zhou C. Investigation of separation force for

constrained-surface stereolithography process from mechanics perspective. Rapid
Prototyping J 2017;23(4).

[8] Jun Wang, Sonjoy Das, Rahul Rai, Chi Zhou, Data-driven simulation for fast
prediction of pull-up process in bottom-up stereo-lithography, Computer-Aided
Design, Volume 99, 2018, 29-42

[9] Chen Y, Zhou C, Lao J. A layerless additive manufacturing process based on cnc
accumulation. Rapid Prototyp J 2011;17(3):218–27.

[10] SLP-4000 solid laser diode plotter. 1997.

[11] Gibson I, Rosen DW, Stucker B. Additive manufacturing technologies. Springer;

2010.

[12] Cao J, Wang D, Zhou Y. Prediction model of rolling force for electrical steel based
on finite element method and neural network. Adv Sci Technol Lett 2014;47:383–6.

[13] Shahani A, Setayeshi S, Nodamaie S, Asadi M, Rezaie S. Prediction of influence

parameters on the hot rolling process using finite element method and neural network.
J Mater Process Technol 2009;209(4):1920–35.

[14] Reddy DY, Pratihar DK. Neural network-based expert systems for predictions of
temperature distributions in electron beam welding process. Int J Adv Manuf Technol
2011;55(5–8):535–48.

31
[15] Hu J, Sundararaman S, Menta V, Chandrashekhara K, Chernicoff W. Failure
pressure prediction of composite cylinders for hydrogen storage using
thermomechanical analysis and neural network. Adv Compos Mater 2009;18(3): 233–
49.

[16] Kazan R, Fırat M, Tiryaki AE. Prediction of springback in wipe-bending process

of sheet metal using neural network. Material Design 2009;30(2):418–23.

[17] Umbrello D, Ambrogio G, Filice L, Shivpuri R. A hybrid finite element method–

artificial neural network approach for predicting residual stresses and the optimal
cutting conditions during hard turning of aisi 52100 bearing steel. Material Design
2008;29(4):873–83.

[18] Arndt O, Barth T, Freisleben B, Grauer M. Approximating a finite element model

by neural network prediction for facility optimization in groundwater engineering.
European J Oper Res 2005;166(3):769–81

[19] Hambli R. Application of neural networks and finite element computation for
multiscale simulation of bone remodeling. J Biomech Eng 2010;132(11): 114502.

[20] Liang L, Liu M, Martin C, Sun W. 2018 A deep learning approach to estimate
stress distribution: a fast and accurate surrogate of finite-element analysis. J. R. Soc.
Interface 15: 20170844.

[21] LeCun Y, Bottou L, Bengio Y, Haffner P. Gradient-based learning applied to

documentrecognition.ProcIEEE1998;86(11):2278–324.

[22] Krizhevsky A, Sutskever I, Hinton GE. Imagenet classification with deep

convolutional neural networks. In:Advances in neural information processing
systems.2012.p.1097–105.

[23] Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D,

Vanhoucke V, Rabinovich A. Going deeper with convolutions. In: Proceedings of the
IEEE conferenceoncomputervisionandpatternrecognition;2015.p.1–9.

[24] A. Toshev and C. Szegedy, "DeepPose: Human Pose Estimation via Deep Neural
Networks," 2014 IEEE Conference on Computer Vision and Pattern Recognition,
Columbus, OH, 2014, pp. 1653-1660.

[25] Simonyan K, Zisserman A. Very deep convolutional networks for large-scale

imagerecognition;2014.ArXivpreprint arXiv:1409.1556.

[26] Mahler J, Liang J, Niyaz S, Laskey M, Doan R, Liu X, Aparicio Ojea J,

Goldberg K, Dex-Net 2.0: Deep Learning to Plan Robust Grasps with Synthetic Point
Clouds and Analytic Grasp Metrics, 2017 ArXiv preprint arXiv:1703.09312

[27] Simonyan K, Zisserman A. Two-Stream Convolutional Networks for Action

Recognition in Videos, 2014 ArXiv preprint arXiv:1406.2199

32