Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Docker all the things! #2

Draft
wants to merge 29 commits into
base: master
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
0079e63
Dockerize all the things
xerootg Jan 10, 2023
974505d
Oops, too greedy with the zune.net rewrite
xerootg Jan 10, 2023
8a8befc
got networking up and running
xerootg Jan 11, 2023
9749fdb
get the whole stack working with the zune HD
xerootg Jan 11, 2023
a037658
get the resources into the publish output
xerootg Jan 11, 2023
2803257
lets pretend we are actually zune.net for a little bit
xerootg Jan 11, 2023
9160bb9
a couple more docker settings
xerootg Jan 11, 2023
d22276a
this doesnt work, but maybe its a starting point
xerootg Jan 11, 2023
23aa541
Somehow, this revives a bunch of the commerce endpoints. You'll have …
xerootg Feb 17, 2023
ac96eea
The RP was not needed, I had an nginx config error, [required] is not…
xerootg Feb 18, 2023
3a473d5
significant container build time improvements by building/publishing …
xerootg Feb 18, 2023
e494d33
Partially working metaservices
xerootg Apr 28, 2023
ff27ec1
A bit of reorganization now that MetaServices is running
xerootg Apr 29, 2023
b2c49d5
adding the ability to lookup the silly int64 id later if we need it
xerootg May 1, 2023
3314d8a
make runProd do important things like recreate the containers. Maybe …
xerootg May 1, 2023
05cb0a3
Handle the full Album search flow for FAI
xerootg May 2, 2023
7a56ac5
More WMIS/FAI stuff
xerootg May 4, 2023
0930bfd
Add automatic track linking, introduce amg-like IDs
xerootg May 5, 2023
9045e4c
splitting the artist controller bits used by the HD out into its own …
xerootg May 26, 2023
77e9aea
split out the HD's image controller methods. Probably should use the …
xerootg May 26, 2023
0275a76
logging is important kids. This time, it was cause i didn't have a va…
xerootg May 26, 2023
3063940
differentiate API versions a bit better
xerootg May 26, 2023
b978f58
Consolidating Image controller, fixing a strange crash in zune.exe wh…
xerootg May 30, 2023
06accc0
- Consolidation of HD specific catalog endpoints
xerootg Jun 6, 2023
8482bf3
Apparently the HD also hits the training data endpoints? COOL!
xerootg Jun 6, 2023
c2f7bfc
make things throw less
xerootg Jun 6, 2023
9b53cf8
implement caching and ratelimiting.
xerootg Jun 6, 2023
498d1cc
10m is rookie numbers.
xerootg Jun 6, 2023
4b26d86
do better. throttle musicbrainz requests or be banned.
xerootg Jun 6, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .dockerignore
Original file line number Diff line number Diff line change
Expand Up @@ -22,4 +22,7 @@
**/secrets.dev.yaml
**/values.dev.yaml
LICENSE
README.md
README.md

#ignore nginx so loadbalancer changes don't trigger a rebuild of the rest of the application
/nginx
4 changes: 4 additions & 0 deletions .editorconfig
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@

# Format: rule = severity # defintion; justification
dotnet_diagnostic.ASP0014.severity = none # Don't use UseEndpoints in trivial APIs; we have some simplistic program.cs, and still need to manually register UseEndpoints
dotnet_diagnostic.CA2254.severity = none # use logging formatter instead of interpolation; formatted strings in logs give me the creeps - also, too many nights oncall sorting impossible to read logs
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@
##
## Get latest from https://github.com/github/gitignore/blob/master/VisualStudio.gitignore

*/DevConstants.cs

# User-specific files
*.rsuser
*.suo
Expand Down
35 changes: 35 additions & 0 deletions .vscode/launch.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
{
"version": "0.2.0",
"configurations": [
{
// Use IntelliSense to find out which attributes exist for C# debugging
// Use hover for the description of the existing attributes
// For further information visit https://github.com/OmniSharp/omnisharp-vscode/blob/master/debugger-launchjson.md
"name": ".NET Core Launch (web)",
"type": "coreclr",
"request": "launch",
"preLaunchTask": "build",
// If you have changed target frameworks, make sure to update the program path.
"program": "${workspaceFolder}/Zune.Net.MetaServices/bin/Debug/net7.0/Zune.Net.MetaServices.dll",
"args": [],
"cwd": "${workspaceFolder}/Zune.Net.MetaServices",
"stopAtEntry": false,
// Enable launching a web browser when ASP.NET Core starts. For more information: https://aka.ms/VSCode-CS-LaunchJson-WebBrowser
// "serverReadyAction": {
// "action": "openExternally",
// "pattern": "\\bNow listening on:\\s+(https?://\\S+)"
// },
"env": {
"ASPNETCORE_ENVIRONMENT": "Development"
},
"sourceFileMap": {
"/Views": "${workspaceFolder}/Views"
}
},
{
"name": ".NET Core Attach",
"type": "coreclr",
"request": "attach"
}
]
}
41 changes: 41 additions & 0 deletions .vscode/tasks.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
{
"version": "2.0.0",
"tasks": [
{
"label": "build",
"command": "dotnet",
"type": "process",
"args": [
"build",
"${workspaceFolder}/Zune.Net.MetaServices/Zune.Net.MetaServices.csproj",
"/property:GenerateFullPaths=true",
"/consoleloggerparameters:NoSummary"
],
"problemMatcher": "$msCompile"
},
{
"label": "publish",
"command": "dotnet",
"type": "process",
"args": [
"publish",
"${workspaceFolder}/Zune.Net.MetaServices/Zune.Net.MetaServices.csproj",
"/property:GenerateFullPaths=true",
"/consoleloggerparameters:NoSummary"
],
"problemMatcher": "$msCompile"
},
{
"label": "watch",
"command": "dotnet",
"type": "process",
"args": [
"watch",
"run",
"--project",
"${workspaceFolder}/Zune.Net.MetaServices/Zune.Net.MetaServices.csproj"
],
"problemMatcher": "$msCompile"
}
]
}
4 changes: 4 additions & 0 deletions Atom/Atom.csproj
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,10 @@
<VersionPrefix>0.1.0</VersionPrefix>
</PropertyGroup>

<PropertyGroup>
<IsPublishable>false</IsPublishable>
</PropertyGroup>

<ItemGroup Condition="$(TargetFramework.StartsWith('netstandard'))">
<PackageReference Include="System.ComponentModel.Annotations" Version="5.0.0" />
<PackageReference Include="System.Xml.ReaderWriter" Version="4.3.1" />
Expand Down
148 changes: 61 additions & 87 deletions Zune.DB.Console/Program.cs
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ class Program
static async Task Main(string[] args)
{
// Set up CLI options
string connectionString = "mongodb://localhost:27017";
string connectionString = "mongodb://root:[email protected]:27017";
string dbName = "Zune";
var options = new OptionSet
{
Expand Down Expand Up @@ -43,25 +43,33 @@ static async Task Main(string[] args)
DatabaseName = dbName,
});

await ctx.ClearAlbumLookupAsync();
return;

await ctx.ClearMembersAsync();
await ctx.ClearTokensAsync();
await ctx.ClearImagesAsync();
// await ctx.ClearTokensAsync();
// await ctx.ClearImagesAsync();

string userName = "[email protected]";
string SID = "S-1-5-21-414912484-GET YOUR OWN PUNK";

// var user = await ctx.GetMemberByName(userName);

string userName = "[email protected]";
var newMember = new Member
{
Updated = DateTime.UtcNow,
Id = Member.GetGuidFromUserName(userName),
SID = SID, // This is my LiveID SID
UserName = userName,
PlayCount = 206,
Xuid = Member.GetXuidFromUserName(userName),
ZuneTag = "YoshiAsk",
DisplayName = "Yoshi Askharoun",
ZuneTag = "xerootg",
DisplayName = "xerootg",
Status = "Reviving the Zune social",
Bio = "A computer science student at Texas A&M Univserity that can't help but bring back dead Microsoft products.",
Location = "College Station, Texas",
UserTile = "http://tiles.zunes.me/tiles/avatar/default.jpg",
Background = "http://tiles.zunes.me/tiles/background/USERBACKGROUND-ART-536X196-49.jpg",
Bio = "resident hacker",
Location = "the internet",
UserTile = "http://tiles.zune.net/tiles/avatar/default.jpg",
Background = "http://tiles.zune.net/tiles/background/USERBACKGROUND-ART-536X196-49.jpg",

AcceptedTermsOfService = true,
AccountSuspended = false,
Expand All @@ -79,92 +87,58 @@ static async Task Main(string[] args)
BillingInstanceId = "6cba2616-c59a-4dd5-bc9e-d41a45215cfa"
};

var tuner = new Tuner
{
Id = "6cba2616-c59a-4dd5-bc9e-d41a45215cfb"
};
newMember.TunerRegisterInfo = tuner;

Guid msgId = Guid.Parse("7e366cd9-6d16-4ddf-9dfe-963acdef4450");
Guid linkId = Guid.Parse("fe9ab096-a072-475b-8e24-0aaacf32852d");
var message = new Message
{
DetailsLink = string.Empty,
Sender = newMember,
Status = "hi",
Wishlist = false,
MediaId = msgId,
Subject = "Microsoft revives long-dead Zune product line",
Received = DateTime.UtcNow,
Type = "message",
Id = msgId.ToString(),
AltLink = new Link
{
Href = "https://rr.noordstar.me/microsoft-revives-long-dead-zune-product--f9628d12",
Id = linkId.ToString()
},
TextContent = "Tech giant Microsoft announced early Monday morning that a new Zune music player is in the works",
};

// Guid msgId = Guid.Parse("7e366cd9-6d16-4ddf-9dfe-963acdef4450");
// Guid linkId = Guid.Parse("fe9ab096-a072-475b-8e24-0aaacf32852d");
// var message = new Message
// {
// DetailsLink = string.Empty,
// Sender = user,
// Status = "hi",
// Wishlist = false,
// MediaId = msgId,
// Subject = "Microsoft revives long-dead Zune product line",
// Received = DateTime.UtcNow,
// Type = "message",
// Id = msgId.ToString(),
// AltLink = new Link
// {
// Href = "https://rr.noordstar.me/microsoft-revives-long-dead-zune-product--f9628d12",
// Id = linkId.ToString()
// },
// TextContent = "Tech giant Microsoft announced early Monday morning that a new Zune music player is in the works",
// };
//newMember.Messages ??= new System.Collections.Generic.List<Message>(1);
//newMember.Messages.Add(message);

Guid badgedId = Guid.Parse("fe9ab096-a072-475b-8e24-0aaacf32852f");
var badge = new Badge
{
Description = "Restore the Zune social",
TypeId = Xml.SocialApi.BadgeType.ActiveForumsBadge_Gold,
Title = "Necromancer",
Image = "https://i.imgur.com/dMwIZs8.png",
MediaId = Guid.NewGuid(),
MediaType = "Application",
//Summary = "Where is this shown? No idea, contact YoshiAsk if you see this in the software"
};
// Guid badgedId = Guid.Parse("fe9ab096-a072-475b-8e24-0aaacf32852f");
// var badge = new Badge
// {
// Description = "Restore the Zune social",
// TypeId = Xml.SocialApi.BadgeType.ActiveForumsBadge_Gold,
// Title = "Necromancer",
// Image = "https://i.imgur.com/dMwIZs8.png",
// MediaId = Guid.NewGuid(),
// MediaType = "Application",
// // Summary = "Where is this shown? No idea, contact YoshiAsk if you see this in the software"
// };

await ctx.CreateAsync(newMember);
//ctx.Messages.Add(message);
//ctx.Tuners.Add(tuner);

userName = "[email protected]";
var member2 = new Member
{
Updated = DateTime.UtcNow,
Id = Member.GetGuidFromUserName(userName),
UserName = userName,
PlayCount = 4123,
Xuid = Member.GetXuidFromUserName(userName),
ZuneTag = "WamWooWam",
DisplayName = string.Empty,
Status = "Restoring Windows Phone 7",
Bio = "he/they, pan, nerd with a strange obsession for windows phone",
Location = "Ireland",
UserTile = "http://i.imgur.com/06BuEKG.jpg",
Background = "http://i.imgur.com/KeZIsxF.jpg",
// var tuner = new Tuner
// {
// Id = "6cba2616-c59a-4dd5-bc9e-d41a45215cfb"
// };
// user.TunerRegisterInfo = tuner;
// user.SID = SID;
// // user.Badges.Add(badge);
// await ctx.UpdateAsync(user);

AcceptedTermsOfService = true,
AccountSuspended = false,
BillingUnavailable = true,
SubscriptionLapsed = true,
TagChangeRequired = false,
UsageCollectionAllowed = false,
ExplicitPrivilege = false,
Lightweight = false,
Locale = "en-GB",
ParentallyControlled = false,
PointsBalance = 100.0,
SongCreditBalance = 0.0,
SongCreditRenewalDate = DateTime.Now.AddDays(1).ToString("O"),
BillingInstanceId = "6cba2616-c59a-4dd5-bc9e-d41a5f215cfa"
};
// System.Console.WriteLine($"Set SID for {user.UserName} to {user.SID}");

tuner = new Tuner
{
Id = "6cba2616-c59a-4dd5-bc9e-d441f315cfb"
};
member2.TunerRegisterInfo = tuner;
//ctx.Messages.Add(message);
// ctx.Tuners.Add(tuner);

await ctx.CreateAsync(member2);
//ctx.Tuners.Add(tuner);
//ctx.SaveChanges();
}
}
}
36 changes: 24 additions & 12 deletions Zune.DB/Helpers.cs
Original file line number Diff line number Diff line change
@@ -1,30 +1,42 @@
using System;
using System.Collections.Generic;
using System.Security.Cryptography;
using System.Text;

namespace Zune.DB
{
public static class Helpers
{
public static Guid GenerateGuid(string content)

private static byte[] GetSha256(string content)
{
// Using MD5 here is fine, as these GUIDs aren't used for
// anything meant to be secure. We could use SHA256, but
// we'd have to crush it down to 16 bytes for the GUID
// anyway. Might as well use MD5 ¯\_(ツ)_/¯
using var hasher = SHA256.Create();
var byteArrayResultOfRawData = Encoding.UTF8.GetBytes(content);

// Compute 128-bit (16-byte) hash
byte[] hash = MD5.HashData(Encoding.UTF8.GetBytes(content));
return new(hash);
return hasher.ComputeHash(byteArrayResultOfRawData);
}

public static Guid GenerateGuid(string content)
{
var result = GetSha256(content);
var guidBase = new byte[16];
for(int i = 0;i < 16; i++)
{
guidBase[i] = result[i];
}
return new(guidBase);
}

public static string Hash(string str)
{
byte[] hash = SHA256.HashData(Encoding.UTF8.GetBytes(str));
var result = GetSha256(str);

string[] hashStr = new string[result.Length];

string[] hashStr = new string[hash.Length];
for (int i = 0; i < hash.Length; i++)
hashStr[i] = hash[i].ToString("X2");
for (int i = 0; i < result.Length; i++)
{
hashStr[i] = result[i].ToString("X2");
}

return string.Join(string.Empty, hashStr).ToUpperInvariant();
}
Expand Down
4 changes: 4 additions & 0 deletions Zune.DB/Models/Member.cs
Original file line number Diff line number Diff line change
Expand Up @@ -42,6 +42,9 @@ public Guid Id

public string UserName { get; set; }

[BsonDefaultValue("NotSet")]
public string SID { get; set; }

public IList<Link> Playlists { get; set; }
public IList<MemberBadge> Badges { get; set; }
public IList<Comment> Comments { get; set; }
Expand Down Expand Up @@ -81,6 +84,7 @@ public Guid Id
public string LastLabelTakedownDate { get; set; }
public Tuner MediaTypeTunerRegisterInfo { get; set; }

// this is supposed to be a list.
public Tuner TunerRegisterInfo { get; set; }

public string UserTile { get; set; }
Expand Down
3 changes: 1 addition & 2 deletions Zune.DB/Models/Tuner.cs
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
using System.Collections.Generic;
using System.ComponentModel.DataAnnotations;
using System.ComponentModel.DataAnnotations;

namespace Zune.DB.Models
{
Expand Down
Loading