api-user
17/07/2018
───────────────────────────
─────────
MASTROMONACO Julien
RainbowBash
42 rue des Arcs-en-ciel
42000 Web Internet (near the mid’ clouds 💔)
1
This document is an alpha version, it can be updated at any time.
Overview
The project is to build an API-REST distributing or receiving user data. This will be responsible
for managing users, different roles, access, and more.
All the basics functions that one could imagine will be made in order to be able to
accommodate changes quickly and cleanly. But, the maleability of the application will allow the
deactivation of lot of "components" without altering the health of the application.
It must be possible to maintain performance constantly, the dispersion of loads on different
servers with monitoring (ELK) is really accurate and should allow the application to respond
very quickly. In the event of server or application failures, redundancy must take over.
Objectives
1. API progressive & Flexible
2. API Scalable
3. API performing 800ms max/request (out of jobs & SSR)
4. Code Quality > 90%
Links
● Trello (https://trello.com/b/sT3QTBc3/user-api)
● Github (https://github.com/JuMastro/user-api)
2
Technologies
Prod
● Ts / Nodejs + Express Framework
● Redis
● PostgresSQL + TypeORM
● MongoDB + Mongoose
● ELK Stack (Elasticsearch, Logstash, Kibana)
● SocketIO (for real-time)
Dev
● Mocha & Chai (Tests)
● Eslint (Linter)
● Jsdoc (Doc generator)
● Supervisor (Restart server on updates or errors)
● Webpack (HTML from PUG)
More
● Docker
● Npm
● TravisCI (v2)
Requirements
● Docker >= 17.0.0
● NodeJS >= 10.0.0
● Npm >= 6.0.0
3
Architecture
4
Requests workflow
Stories
Users
First stage
● Registration ● Profile update
● Registration confirmation ● Email update
● Resend registration confirmation ● Email update confirmation
● Login ● Password update
● Password forgot ● Get my public tokens
● Password reset ● Create public token
● Get profile data ● Delete public token
● Username update
Second stage
● Disable account ● Get own data
5
Administrators
First stage
● Add roles ● Update an individual user
● Remove roles ● Update users by selecting
● Get users list (small dataset) ● Have filters for users list
● Get individual user data ● Create account
Second stage
● Disable accounts ● Get user data set
● Delete accounts ● Get anonymous user data set
● Get disabled users ● Get statistics for each use case of
● Send a newsletter application
● Add group ● Get monitoring
● Remove group ● Reset user public_key
● Update group ● Add user public_token
● Add user to group ● Remove user public_token
● Remove user to group ● Ban user or ip
● Get banned users
● Get banned ip
6
Developers
First stage
● Structured app tree ● Translation module
● Docker configurations ● PUG to HTML script
● Globals configurations ● Response (external)
● Server running ● ErrorClass response (internal)
● Load balancer app ● SQL (entities schemas abstracts)
● Middleware body-parser ● noSQL (entities schemas abstracts)
● Middleware requestHandler ● SQL Repository
● Middleware ip-access ● noSQL Repository
● Middleware (auth) user-access ● Fixtures SQL
● Define roles hierarchy ● Fixtures NoSQL
● Abstract Controller ● SQL expected errors handler
● Routing module ● Mailer module using nodemailer
● Public keys generator ● Hasher module using Argon2i
● Tokens generator ● JsonWebToken provider module
● Identifiant key generator (uuid) ● Data validator module
● Data formatter & anonymizer (recipe-consumer, izitjs rewriting)
Second stage
● ELK Stack dockerized ● Redis dockerized
● Elasticsearch config & module ● Redis Cluster Adapter
● Logstash config & module ● Client tracker
● Kibana config & module
7
8
email regex(email)
Norms
Naming Conventions
variables camelCase var infoUser = {}
functions infoUser.js
function files
For more norms, follow the linter configuration.
9
Specs
Errors
If an error is detected there are two possible cases, first if the error is an expected error then
the route will return the error object with error code 400 (Bad request).In the second case, the
one where the error is not an expected error, the route will console.log the error and return an
error code 500 (Internal error).
Routes
POST PUBLIC /registration Register an user
GET PUBLIC /registration/{token} Confirm an user registration
POST PUBLIC /registration-confirmation-reset/{token} Resend an user registration confirmation
POST PUBLIC /login Log to an account
POST PUBLIC /password-forgot Send a recovery password email
POST PUBLIC /password-reset Redefine a password
GET User /account/profile See current logged account profile
POST User /account/profile Update user profile
POST User /account/username Update username
POST User /account/email Update email
GET User /account/email-confirmation Confirm new email
POST User /account/password Update password
GET User /account/tokens Get current user tokens
POST User /account/tokens Create new token
GET User /account/tokens/delete/{tokenId} Delete user token
GET User /account/disable Disable user account
GET User /account/disable/{token} Disable account confirmation
GET User /account/get-my-data Get current user dataset
10
POST Admin /admin/roles/add Add roles to an user
POST Admin /admin/roles/delete Delete roles to an user
GET Admin /admin/users/{userId} Get user data
GET Admin /admin/users Get users data list
POST Admin /admin/users/{userId} Update an user
POST Admin /admin/users Update an users list
POST Admin /admin/users/create Create an user
POST Admin /admin/users/disable Disable an users list
POST Admin /admin/users/delete Delete an users list
GET Admin /admin/users/is-disabled Get disabled users
POST Admin /admin/newsletter Send a newsletter
GET Admin /admin/users/{userId}/reset-public-key Reset an user public key
POST Admin /admin/users/public-token Create an user public token
GET Admin /admin/users/public-token/{tokenId}/delete Remove an user public token
POST Admin /admin/users/ban Ban an user
POST Admin /admin/ip/ban Ban an ip
GET Admin /admin/users/banned Get banned users
GET Admin /admin/ip/banned Get banned ip
GET Admin /admin/users/{userId}/dataset Download an user dataset
GET Admin /admin/users/{userId}/dataset?anonymous=true Download anonymized user dataset
POST Admin /admin/groups Create a group
GET Admin /admin/groups/{groupId}/delete Delete a group
POST Admin /admin/groups/{groupId} Update a group
GET Admin /admin/groups Get list of groups
POST Admin /admin/groups/add-user Add an user to a group
11
Developers
Routes
No developper route needed...
Modules
13
V. Load balancer
4H
NO_REQUIREMENT
● Make express load-balancer app
● Make tests
WORKFLOW:
VI. Middleware body-parse
30Min
NO_REQUIREMENT
● Make body-parser middleware module
● Plug middleware module to app
WORKFLOW:
14
15
X. Abstract Controller
1H
NO_REQUIREMENT
● Write an abstract controller with differents “commons methods & modules”
XXV. Fixtures NoSQL
3H
NO_REQUIREMENT
● Make fixtures for each nosql schemas
Users
Modules
No user module needed...
22
Routes
I. Registration
3-4 H
PUBLIC_KEY_GENERATOR, TOKEN_GENERATOR, EXTERNAL_ID_GENERATOR
● Create route configuration file and register it to app
● Create email registration confirmation template
● Create controller.action with W
ORKFLOW
● Create tests
DETAILS:
Route path /registration
Method POST: Public
Controller action name SecurityController: registration
Post data username, email, password, passwordConfirmation
WORKFLOW:
23
db.userInfo.create({
last_user_ip: *UserIP*,
last_user_ip_date: Date(now),
registration_date: Date(now)
})
db.userProfile.create({
*EMPTY*
})
db.userArchive.create({
olds_usernames: [],
olds_emails: [],
olds_public_key: [],
user_id: *UserID*
})
db.userSecurity.create({
public_key: *generated*,
account_confirmation_token: *generated*,
account_confirmation_token_limit: Date(now+1H)
})
db.user.create({
identifiant_key: *generated*,
username: validate(*input*),
email: validate(*input*),
password: hash(validate(*input*)),
roles: [‘ROLE_USER’],
groups: [],
is_active: false,
is_blocked: false,
is_disabled: false,
is_confirmed: false,
user_security_id: *UserSecurityID*,
user_profile_id: *UserProfileID*,
user_info_id: *UserInfoID*
})
24
db.userInfo.update({
registration_confirmation_date: Date(now)
})
db.user.update({
is_active: true,
is_confirmed: true
})
db.userSecurity.update({
account_confirmation_token: null,
account_confirmation_token_limit: null
})
25
DETAILS:
Route path /registration-confirmation-reset/{token}
Method POST: Public
Controller action name SecurityController: resendAccountConfirmaton
Post data email
WORKFLOW:
db.userSecurity.update({
account_confirmation_token: *generated*,
account_confirmation_token_limit: Date(now + 1H)
})
26
IV. Login
3 H
HASHER, JWT_ADAPTER
● Create route configuration file and register it to app
● Create controller.action with W
ORKFLOW
● Create tests
DETAILS:
Route path /login
Method POST: Public
Controller action name SecurityController: login
Post data email | username, password
WORKFLOW:
db.userInfo.update({
last_conneted_date: Date(now)
})
27
V. Password forgot
3 H
TOKEN_GENERATOR
● Create route configuration file and add it to app
● Create email template
● Create controller.action with W
ORKFLOW
● Create route tests
DETAILS:
Route path /forgot-password
Method POST: Public
Controller action name SecurityController: forgotPassword
Post data email
WORKFLOW:
db.userSecurity.update({
reset_password_token: *generated*,
reset_password_token_limit: *generated*
})
28
WORKFLOW:
db.user.update({
password: *hash(validate(input))*,
reset_password_token: null,
reset_password_token_limit: null
})
db.userInfo.update({
last_password_update: Date(now)
})
29
WORKFLOW:
DETAILS:
Route path /account/profile
Method GET: Protected(Auth-User)
Controller action name UserController: getProfile
Get data
30
db.userProfile.update({ *PROFILE_DATA* })
db.userInfos.update({
last_profile_update: Date(now)
})
31
db.user.update({
username: *validate(input)*
})
db.userArchive.update({
olds_username: exist.push(old_username)
})
db.userInfos.update({
last_username_update_date: Date(now)
})
32
X. Email update
3 H
TOKEN_GENERATOR, HASHER
● Create route configuration file and add it to app
● Create email template
● Create controller.action with W
ORKFLOW
● Create route tests
DETAILS:
Route path /account/email
Method POST: Private(Auth-User)
Controller action name SecurityController: updateEmail
Post data email, password
WORKFLOW:
33
db.user.update({
email: *validate(input)*,
is_confirmed: false
})
db.userArchive.update({
olds_email: exists.push(old_email)
})
db.userInfos.update({
last_email_update: Date(now)
})
db.userSecurity.update({
update_email_token: *generated*,
update_email_token_limit: Date(now + 1H)
})
34
db.user.update({
is_confirmed: true
})
db.userSecurity.update({
update_email_token: null,
update_email_token_limit: null
})
35
DETAILS:
Route path /account/password
Method POST: Private(Auth-User)
Controller action name SecurityController: updatePassword
Get data oldPassword, password, passwordConfirmation
db.user.update({
password: *hash(validate(input))*
})
db.userInfo.update({
last_password_update: Date(now)
})
36
37
WORKFLOW:
db.publicTokens.create({
secure_token: *generated*,
description: validate(input),
date: Date(now),
date_limit: Date(now + 120D),
user_id: *UserID*
})
38
WORKFLOW:
db.publicTokens.delete(WHERE token = GET[token])
39
db.userSecurity.update({
disable_account_token: *generated*,
disable_account_token_limit: Date(now)
})
40
XVII. Disable account confirmation
3 H
TOKEN_GENERATOR
● Create route configuration file and add it to app
● Create email templates
● Create controller.action with W
ORKFLOW
● Create route tests
DETAILS:
Route path /account/disable/{token}
Method GET: Private(Auth-User)
Controller action name SecurityController: disableAccountConfirmation
Get data token
WORKFLOW:
db.user.update({
is_disabled: true
})
db.userSecurity.update({
disable_account_token: null,
disable_account_token_limit: null
})
41
XVIII. Get own data
3 H
DATA_FORMATTER
● Create route configuration file and add it to app
● Create email templates
● Create controller.action with W
ORKFLOW
● Create route tests
DETAILS:
Route path /account/get-my-data
Method GET: Private(Auth-User)
Controller action name UserController: getMyOwnData
Get data
WORKFLOW:
42
Administrators
Routes
I. Add roles
2 H
NO_REQUIREMENT
● Create route configuration file and add it to app
● Create controller.action with W
ORKFLOW
● Create route tests
DETAILS:
Route path /admin/roles/add
Method POST: Private(Auth-Admin)
Controller action name AdminManagementController: addRoles
Post data userId, [rolesList]
WORKFLOW:
db.user.update({
roles: exists.merge(input)
})
43
db.user.update({
roles: exists.remove(inputs)
})
44
45
46
47
48
49
db.userInfo.create({
last_user_ip: *UserIP*,
last_user_ip_date: Date(now),
registration_date: Date(now),
registration_confirmation_date: Date(now)
})
db.userProfile.create({
*EMPTY*
})
db.userArchive.create({
olds_usernames: [],
olds_emails: [],
user_id: *UserID*
})
db.userSecurity.create({
public_key: *generated*,
account_confirmation_token: null,
account_confirmation_token_limit: null
})
db.user.create({
identifiant_key: *generated*,
username: validate(*input*),
email: validate(*input*),
password: hash(validate(*input*)),
roles: [‘ROLE_USER’],
groups: [],
is_active: false,
is_blocked: false,
is_disabled: false,
is_confirmed: false,
user_security_id: *UserSecurityID*,
user_profile_id: *UserProfileID*,
user_info_id: *UserInfoID*
})
50
db.each.userInfo.update({
reason_disable: *input*
})
db.each.user.update({
is_disabled: true
})
51
db.each.user.delete( * CASCADE * )
52
54
db.userSecurity.update({
public_key: *generated*
})
db.userArchive.update({
olds_public_key: exist.push(input)
})
55
db.userPublicTokens.create({
secure_token: *generated*,
description: *validate(input)*,
date: Date(now),
date_limit Date(now + 120D),
user_id: *UserID*
})
56
db.userPublicTokens.create(WHERE id = :tokenId)
57
db.user.update({
58
is_active: false,
is_blocked: true
})
db.userSecurity.update({
blocked_account_date: Date(now),
blocked_account_limit: Date(now + input)
})
db.userInfo.update({
reason_blocked: validate(input)
})
59
db.blockedIp.create({
ip: validate(input),
type: validate(input),
reason: validate(input),
date: Date(now),
date_limit: Date=(input)
})
60
61
62
63
64
db.group.create({
name: validate(input),
description: validate(input),
created_date: Date(now)
})
65
db.group.remove(WHERE group_id = :id)
66
db.group.update({
name: validate(input),
description: validate(input),
updated_date: Date(now)
})
67
68
db.usersGroups.create({
user_id: validate(input),
group_id: validate(input),
joinded_date: Date(now)
})
69
db.usersGroups.remove(
WHERE user_id = userId,
group_id = groupId
)
70
Module
I. Have filters for users list
8 H
ADMINISTRATORS_ROUTES
● Create module
● Plug module to app
● Create module tests
The purpose of this module is to consume the parameters of the query of the route to see the
list of users. As the route is only accessible by admins, a large number of parameters can serve
as filters, starting with the global dates, roles, groups, username and more.
More details on S
toriesBoard