Version 2.0 of Threat Dragon

[jgadsden]

There are some fundamental changes that we need to do to the Threat Dragon code because of the problem we have with the drawing package JointJs. This is at the core of the project, so we should look at revising the structure of the whole project.

We have a roadmap for version 2.0 and @lreading has started on some of this with pull request #67 . We maybe should discuss how we get to version 2.0, and also why we think it is necessary

[jgadsden]

Reasons for version 2.0

• JointJS is not providing the features that we need such as copy and paste, drag, zoom and pan. They are provided in the pay-for version but not the free version that we use, and so using JointJS is not viable in the long term
• the code base is a steep learning curve. We have 3 repos (webapp, desktop and core) that is difficult to swap between
• some of the technology is not what we would use if the project was begun today. For example we would probably use React or Vue rather than Angular
• we should consider using mxgraph for the drawing package, seems to have everything we want

Having said all that, maximum kudos to @mike-goodwin for creating something seven years ago that is used by a large community and is still in active development. A testament to the good principles it was based on - accessibility, simplicity and fulfilling a need

[jgadsden]

There are two approaches to version 2.0, we could add to the existing version 1.x and get there step by step or we could start afresh for 2.0 and reuse parts of 1.x as we need. I am OK with both approaches, and I created a couple of experiments to see what the final code base might look like:

• threat dragon framework with docs as a sub project and provision for API
• desktop app which has threat dragon as a sub project

[lreading]

I see benefits to both approaches.

Some benefits to keeping the project structure and iteratively developing/refactoring:

• Main trunk is constantly getting updates and is in a deployable state
• Less code rot
• Any new PRs that are opened by the community for enhancements, bugfixes, etc are in the actively developed branch, will not need as much refactoring
• Less likely to accidentally lose a feature in transition

Advantages of starting "from scratch" (mostly):

• Opportunity to establish clear architectural decisions
• Overhaul of all features, documentation, etc etc
• Possibly a faster turn-around
• All the new shiny things (only somewhat sarcastic, it is really nice being able to work with the latest and greatest tech!)

With all of that in mind, I think I would have a slight preference to refactoring it and getting it to where we want to go that way as opposed to starting fresh. Based on the roadmap, I think the approach that I would take would look something like this:

1. Separate front end from back-end (mostly done, still a shared package.json which is fine, but we should consider if we want to keep it that way or not)
2. Integrate babel or webpack/babel for the back-end code to allow for ES6 code
3. Refactor back-end to use es6
4. Update jshint to alert on non-es6 (var vs let/const, etc)
5. Add sinon/chai (or chosen tech) for back-end testing, refactor existing tests
6. Work on authentication/storage methods, create a pattern to make it easier to add future ones
7. Add a local storage solution (this allows for an easier demo and e2e tests)
8. Flesh out authentication and patterns for an API
9. Use apidoc or similar to auto-generate API documentation, which maybe should be separate from user documentation? Or maybe it should be in the same place?
10. Add a new directory in the root for the 2.0 UI, start fresh with whatever check is chosen (Vue, react). Work on this while leaving the "legacy" ui to ensure that it's always in a usable state
11. Start writing e2e tests as features are completed in the new UI
12. At a certain point, remove the legacy UI and move forward only with the new UI

This is a lot of work and I'm sure there would be some challenges along the way that I'm not seeing. That is only for the architecture of the project, never mind the new features that need to be added.

Bearing in mind that I'm brand new to this project, I don't want anyone to think I'm trying to take over. These are only my ideas, and I'm just here to help however I can. I'm prepared to commit a lot of time to this project, and am eager to continue contributing! Thanks again for all of the hard work that has made this project a fantastic open-source tool!

[jgadsden]

OK, I agree and lets go with a refactoring approach rather than start again. Seems a safer route and also changes can be more immediate available.

When we swap out JointJS for mxgraph then that could be the point that it is version 2.0

And great to have @lreading contributing, much appreciated

[lreading]

Taking a quick peek at the desktop repo, it does seem like there's a lot of duplicated code between the repos. If we had all the code in a single repo, we could have a single source of truth and ensure everything is up to date as opposed to managing multiple versions of multiple projects. It could even allow us to create a PR checklist to ensure that all related components are being updated (docs, test desktop app, etc), as well as a better opportunity to automate some stuff (continuous delivery maybe).

Thoughts?

[jgadsden]

Yes, agreed, I think we need to work towards combining most of everything back into Threat Dragon repo.

It was split off in 2016 to create the desktop version, but now we would include threat dragon as a submodule to the threat dragon desktop repo, and build the desktop application from that