Skip to content

Commit

Permalink
Merge pull request #140 from DrewNaylor/new-virtual-mode-with-package…
Browse files Browse the repository at this point in the history
…-class

Significantly-faster package list loading and nearly-instant search, both provided by a DataView and its RowFilter.

Thought this was going to be using VirtualMode and a Package class, but so far these changes are major improvements, so I might not do those unless necessary.

Current known bugs and issues:
- [x] The database isn't being closed after updating the cache, so it says that it's in use when I try to update again. (This should be fixed now, I hope. Mainly what I did was to have it clear the pools just before returning the datatable so that hopefully it's not in memory now.)
- [x] There aren't dropdowns/comboboxes in the `Actions` column right now, as I don't entirely know how to put them into DataTables. This is super important to ensure people can use it anyway they want. Edit: maybe I can just have it so that people can press `spacebar` when the `Action` column is the current column and open a makeshift menu for the `Action` items that's used instead of the previous dropdown. Edit 2: I just added the `Ctrl+M` keyboard shortcut so people can open the context menu in the top-left corner of the package list DataGridView, which is a lot less work and should be better than what it was before. People can still use `Alt+S` if they wish to open the `Selected packages` menu, which has the `Ctrl+M` button in it. Only potential issue is this'll cover up some of the packages, but for now, I don't have any other simple solutions that aren't more buggy than the previous dropdowns.
- [x] `Actions` and `Status` columns aren't auto-sized correctly.
- Not really an issue, but there may be a lot of useless code that was necessary previously that doesn't present any noticeable problems that'll have to be removed.
- TODO: I need to have a column for `Installed version` once there's a way to allow users to choose a different version to install rather than the latest version. This'll just be something I'll add in the future, rather than in this PR.

Additions:
- New setting: `DebuggingShowManifestPathColumn`, Boolean, defaults to `False`. You can use this to show the ManifestPath column, as expected.
- Since there's a way to know how many rows are currently being shown vs how many are loaded in total, the statusbar now displays the number of listed packages alongside the total number of loaded packages. This is sorta like what Synaptic does, except the number of packages marked in various ways isn't shown yet. Once I can get that working, I'll add that feature.
- Column widths are now saved across refreshing the cache, which is effectively the same thing that happened last time, except this is necessary as we're removing everything from the datagridview on cache updates.

Fixes:
- Selecting all packages with Ctrl+A while searching will select every package, even the ones that aren't visible. (fixes #13)
- Last package selected displays its details even if there are no search results. (fixes #17, though I did push a commit earlier with the "fixes" thing)

Removals:
- Forgot about the ManifestType column, but I didn't add support for it in the new code. Copied over the commented-out code so it can be used if necessary, though it'll require changes.
- We're not using the `PackageInfo` class or whatever it's called sometimes anymore, so I removed it completely and it's not just commented out now.

Other changes:
- Descriptions and manifest paths are loaded at the same time as the rest of the details are loaded from the database to make using the DataTable easier/possible. I decided to make the code that adds the rows to the datatable use Await Task.Run to try to speed it up, and it may help a little, but I'm not sure how much.
- Libraries were updated.
- Just so I don't miss this when writing the changelog, searches are now near-instant, and loading the package list is usually way faster, even on my desktop which is a Ryzen 5800X. I think it also uses a lot less RAM now too, usually staying under 80 MB of RAM, whereas before it would easily go into 120 MB or more. At least, that was based on the highest number in the memory usage graph in Visual Studio, so it's not exact. Sometimes it can use a lot more, too, so I need to figure out how to keep the memory usage lower. Running without the debugger makes package list loading faster, though rarely it's still really slow for some reason.
- Instead of having the progressbar go up by 1 for each package, it now goes up every 100 packages when there are 100 or more in the list, goes up by 10 if there are 10 or more but fewer than 99 packages, and up by 1 if there are fewer than 10 packages. This may help improve performance by not constantly updating the UI thread.
- Checking to see how many manifests are available is now done by using `If ManifestPaths.Count = 0...` instead of `If ManifestPaths(0) = String.Empty` to make sure it doesn't break as easily.
- Loading from the Sqlite database into a DataTable is now done Async to hopefully not make things lock up as much. In my testing, I have only noticed guinget's titlebar say "(Not Responding)" once or twice, though there were a lot of times Task Manager said it wasn't responding, but most people probably wouldn't have Task Manager open and looking at guinget's process in particular.
- HiDPI Mode now hides the package list and shows it again after changing the height for each row. This should help improve performance, though it's still slow, so I still recommend toggling it before loading the package list.
- Instead of using `AllCells` for the autosizemode for the `Action` and `Status` columns, they're now using `DisplayedCells`. Hopefully this speeds things up.
  • Loading branch information
DrewNaylor authored Jan 24, 2022
2 parents 56cf2a3 + d566f33 commit b2549b4
Show file tree
Hide file tree
Showing 11 changed files with 481 additions and 296 deletions.
9 changes: 6 additions & 3 deletions guinget/App.config
Original file line number Diff line number Diff line change
Expand Up @@ -89,22 +89,25 @@
<setting name="UseKDEStyleFastResize" serializeAs="String">
<value>True</value>
</setting>
<setting name="DebuggingShowManifestPathColumn" serializeAs="String">
<value>False</value>
</setting>
</guinget.My.MySettings>
</userSettings>
<runtime>
<assemblyBinding xmlns="urn:schemas-microsoft-com:asm.v1">
<probing privatePath="lib" />
<dependentAssembly>
<assemblyIdentity name="SQLitePCLRaw.core" publicKeyToken="1488e028ca7ab535" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-2.0.4.976" newVersion="2.0.4.976" />
<bindingRedirect oldVersion="0.0.0.0-2.0.7.1395" newVersion="2.0.7.1395" />
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="System.Runtime.CompilerServices.Unsafe" publicKeyToken="b03f5f7f11d50a3a" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-5.0.0.0" newVersion="5.0.0.0" />
<bindingRedirect oldVersion="0.0.0.0-6.0.0.0" newVersion="6.0.0.0" />
</dependentAssembly>
<dependentAssembly>
<assemblyIdentity name="Microsoft.Data.Sqlite" publicKeyToken="adb9793829ddae60" culture="neutral" />
<bindingRedirect oldVersion="0.0.0.0-5.0.7.0" newVersion="5.0.7.0" />
<bindingRedirect oldVersion="0.0.0.0-6.0.1.0" newVersion="6.0.1.0" />
</dependentAssembly>
</assemblyBinding>
</runtime>
Expand Down
71 changes: 44 additions & 27 deletions guinget/MainWindow.Designer.vb

Large diffs are not rendered by default.

491 changes: 314 additions & 177 deletions guinget/MainWindow.vb

Large diffs are not rendered by default.

14 changes: 13 additions & 1 deletion guinget/My Project/Settings.Designer.vb

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

3 changes: 3 additions & 0 deletions guinget/My Project/Settings.settings
Original file line number Diff line number Diff line change
Expand Up @@ -74,5 +74,8 @@
<Setting Name="UseKDEStyleFastResize" Type="System.Boolean" Scope="User">
<Value Profile="(Default)">True</Value>
</Setting>
<Setting Name="DebuggingShowManifestPathColumn" Type="System.Boolean" Scope="User">
<Value Profile="(Default)">False</Value>
</Setting>
</Settings>
</SettingsFile>
28 changes: 14 additions & 14 deletions guinget/guinget.vbproj
Original file line number Diff line number Diff line change
Expand Up @@ -103,20 +103,20 @@
<Reference Include="libscrollswitchtabs">
<HintPath>..\libs\libscrollswitchtabs.dll</HintPath>
</Reference>
<Reference Include="Microsoft.Data.Sqlite, Version=5.0.7.0, Culture=neutral, PublicKeyToken=adb9793829ddae60, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.Data.Sqlite.Core.5.0.7\lib\netstandard2.0\Microsoft.Data.Sqlite.dll</HintPath>
<Reference Include="Microsoft.Data.Sqlite, Version=6.0.1.0, Culture=neutral, PublicKeyToken=adb9793829ddae60, processorArchitecture=MSIL">
<HintPath>..\packages\Microsoft.Data.Sqlite.Core.6.0.1\lib\netstandard2.0\Microsoft.Data.Sqlite.dll</HintPath>
</Reference>
<Reference Include="SQLitePCLRaw.batteries_v2, Version=2.0.4.976, Culture=neutral, PublicKeyToken=8226ea5df37bcae9, processorArchitecture=MSIL">
<HintPath>..\packages\SQLitePCLRaw.bundle_e_sqlite3.2.0.4\lib\net461\SQLitePCLRaw.batteries_v2.dll</HintPath>
<Reference Include="SQLitePCLRaw.batteries_v2, Version=2.0.7.1395, Culture=neutral, PublicKeyToken=8226ea5df37bcae9, processorArchitecture=MSIL">
<HintPath>..\packages\SQLitePCLRaw.bundle_e_sqlite3.2.0.7\lib\net461\SQLitePCLRaw.batteries_v2.dll</HintPath>
</Reference>
<Reference Include="SQLitePCLRaw.core, Version=2.0.4.976, Culture=neutral, PublicKeyToken=1488e028ca7ab535, processorArchitecture=MSIL">
<HintPath>..\packages\SQLitePCLRaw.core.2.0.4\lib\netstandard2.0\SQLitePCLRaw.core.dll</HintPath>
<Reference Include="SQLitePCLRaw.core, Version=2.0.7.1395, Culture=neutral, PublicKeyToken=1488e028ca7ab535, processorArchitecture=MSIL">
<HintPath>..\packages\SQLitePCLRaw.core.2.0.7\lib\netstandard2.0\SQLitePCLRaw.core.dll</HintPath>
</Reference>
<Reference Include="SQLitePCLRaw.nativelibrary, Version=2.0.4.976, Culture=neutral, PublicKeyToken=502ed628492ab262, processorArchitecture=MSIL">
<HintPath>..\packages\SQLitePCLRaw.bundle_e_sqlite3.2.0.4\lib\net461\SQLitePCLRaw.nativelibrary.dll</HintPath>
<Reference Include="SQLitePCLRaw.nativelibrary, Version=2.0.7.1395, Culture=neutral, PublicKeyToken=502ed628492ab262, processorArchitecture=MSIL">
<HintPath>..\packages\SQLitePCLRaw.bundle_e_sqlite3.2.0.7\lib\net461\SQLitePCLRaw.nativelibrary.dll</HintPath>
</Reference>
<Reference Include="SQLitePCLRaw.provider.dynamic_cdecl, Version=2.0.4.976, Culture=neutral, PublicKeyToken=b68184102cba0b3b, processorArchitecture=MSIL">
<HintPath>..\packages\SQLitePCLRaw.provider.dynamic_cdecl.2.0.4\lib\netstandard2.0\SQLitePCLRaw.provider.dynamic_cdecl.dll</HintPath>
<Reference Include="SQLitePCLRaw.provider.dynamic_cdecl, Version=2.0.7.1395, Culture=neutral, PublicKeyToken=b68184102cba0b3b, processorArchitecture=MSIL">
<HintPath>..\packages\SQLitePCLRaw.provider.dynamic_cdecl.2.0.7\lib\netstandard2.0\SQLitePCLRaw.provider.dynamic_cdecl.dll</HintPath>
</Reference>
<Reference Include="System">
<HintPath>..\..\..\..\..\..\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.6.1\System.dll</HintPath>
Expand All @@ -135,8 +135,8 @@
<Reference Include="System.Numerics.Vectors, Version=4.1.4.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
<HintPath>..\packages\System.Numerics.Vectors.4.5.0\lib\net46\System.Numerics.Vectors.dll</HintPath>
</Reference>
<Reference Include="System.Runtime.CompilerServices.Unsafe, Version=5.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
<HintPath>..\packages\System.Runtime.CompilerServices.Unsafe.5.0.0\lib\net45\System.Runtime.CompilerServices.Unsafe.dll</HintPath>
<Reference Include="System.Runtime.CompilerServices.Unsafe, Version=6.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a, processorArchitecture=MSIL">
<HintPath>..\packages\System.Runtime.CompilerServices.Unsafe.6.0.0\lib\net461\System.Runtime.CompilerServices.Unsafe.dll</HintPath>
</Reference>
<Reference Include="System.Windows.Forms" />
<Reference Include="System.Xml" />
Expand Down Expand Up @@ -272,12 +272,12 @@
call "$(SolutionDir)postbuild.bat"
REM call "$(SolutionDir)postbuild64.bat"</PostBuildEvent>
</PropertyGroup>
<Import Project="..\packages\SQLitePCLRaw.lib.e_sqlite3.2.0.4\build\net461\SQLitePCLRaw.lib.e_sqlite3.targets" Condition="Exists('..\packages\SQLitePCLRaw.lib.e_sqlite3.2.0.4\build\net461\SQLitePCLRaw.lib.e_sqlite3.targets')" />
<Import Project="..\packages\SQLitePCLRaw.lib.e_sqlite3.2.0.7\build\net461\SQLitePCLRaw.lib.e_sqlite3.targets" Condition="Exists('..\packages\SQLitePCLRaw.lib.e_sqlite3.2.0.7\build\net461\SQLitePCLRaw.lib.e_sqlite3.targets')" />
<Target Name="EnsureNuGetPackageBuildImports" BeforeTargets="PrepareForBuild">
<PropertyGroup>
<ErrorText>This project references NuGet package(s) that are missing on this computer. Use NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file is {0}.</ErrorText>
</PropertyGroup>
<Error Condition="!Exists('..\packages\SQLitePCLRaw.lib.e_sqlite3.2.0.4\build\net461\SQLitePCLRaw.lib.e_sqlite3.targets')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\SQLitePCLRaw.lib.e_sqlite3.2.0.4\build\net461\SQLitePCLRaw.lib.e_sqlite3.targets'))" />
<Error Condition="!Exists('..\packages\SQLitePCLRaw.lib.e_sqlite3.2.0.7\build\net461\SQLitePCLRaw.lib.e_sqlite3.targets')" Text="$([System.String]::Format('$(ErrorText)', '..\packages\SQLitePCLRaw.lib.e_sqlite3.2.0.7\build\net461\SQLitePCLRaw.lib.e_sqlite3.targets'))" />
</Target>
<!-- To modify your build process, add your task inside one of the targets below and uncomment it.
Other similar extension points exist, see Microsoft.Common.targets.
Expand Down
14 changes: 7 additions & 7 deletions guinget/packages.config
Original file line number Diff line number Diff line change
@@ -1,13 +1,13 @@
<?xml version="1.0" encoding="utf-8"?>
<packages>
<package id="Microsoft.Data.Sqlite" version="5.0.7" targetFramework="net48" />
<package id="Microsoft.Data.Sqlite.Core" version="5.0.7" targetFramework="net48" />
<package id="SQLitePCLRaw.bundle_e_sqlite3" version="2.0.4" targetFramework="net48" />
<package id="SQLitePCLRaw.core" version="2.0.4" targetFramework="net48" />
<package id="SQLitePCLRaw.lib.e_sqlite3" version="2.0.4" targetFramework="net48" />
<package id="SQLitePCLRaw.provider.dynamic_cdecl" version="2.0.4" targetFramework="net48" />
<package id="Microsoft.Data.Sqlite" version="6.0.1" targetFramework="net48" />
<package id="Microsoft.Data.Sqlite.Core" version="6.0.1" targetFramework="net48" />
<package id="SQLitePCLRaw.bundle_e_sqlite3" version="2.0.7" targetFramework="net48" />
<package id="SQLitePCLRaw.core" version="2.0.7" targetFramework="net48" />
<package id="SQLitePCLRaw.lib.e_sqlite3" version="2.0.7" targetFramework="net48" />
<package id="SQLitePCLRaw.provider.dynamic_cdecl" version="2.0.7" targetFramework="net48" />
<package id="System.Buffers" version="4.5.1" targetFramework="net48" />
<package id="System.Memory" version="4.5.4" targetFramework="net48" />
<package id="System.Numerics.Vectors" version="4.5.0" targetFramework="net48" />
<package id="System.Runtime.CompilerServices.Unsafe" version="5.0.0" targetFramework="net48" />
<package id="System.Runtime.CompilerServices.Unsafe" version="6.0.0" targetFramework="net48" />
</packages>
97 changes: 55 additions & 42 deletions libguinget/PackageListTools.vb
Original file line number Diff line number Diff line change
Expand Up @@ -234,17 +234,17 @@ Public Class PackageListTools
End If
' Now download the database.
If IO.Directory.Exists(DatabaseTempDir) Then
' Exists; re-create it.
If Await DeleteTempDirAsync("winget-db", True) = False Then
' If there's an issue and a file is open, stop updating.
Exit Function
End If
ElseIf Not IO.Directory.Exists(DatabaseTempDir) Then
' Doesn't exist; create it.
Await Task.Run(Sub()
System.IO.Directory.CreateDirectory(DatabaseTempDir)
End Sub)
' Exists; re-create it.
If Await DeleteTempDirAsync("winget-db", True) = False Then
' If there's an issue and a file is open, stop updating.
Exit Function
End If
ElseIf Not IO.Directory.Exists(DatabaseTempDir) Then
' Doesn't exist; create it.
Await Task.Run(Sub()
System.IO.Directory.CreateDirectory(DatabaseTempDir)
End Sub)
End If
' Now download.
Await DownloadPkgListWithProgressAsync("https://winget.azureedge.net/cache/source.msix",
"winget-db")
Expand Down Expand Up @@ -426,11 +426,11 @@ Public Class PackageListTools
"Extracting manifests")
End Try
Else
' The calling app wants to use 7zip, so use it.
' Make sure it doesn't have "://" in the path.
If PathTo7zip.Contains("://") Then
PathTo7zip = "C:\Program Files\7-Zip\7z.exe"
End If
' The calling app wants to use 7zip, so use it.
' Make sure it doesn't have "://" in the path.
If PathTo7zip.Contains("://") Then
PathTo7zip = "C:\Program Files\7-Zip\7z.exe"
End If

Using extraction7z As New Process
Await Task.Run(Sub()
Expand All @@ -455,11 +455,11 @@ Public Class PackageListTools
End Using
End If

'MessageBox.Show("7zip finished")
'MessageBox.Show("7zip finished")



End Using
End Using

'MessageBox.Show("Done extracting.")

Expand Down Expand Up @@ -530,17 +530,17 @@ Public Class PackageListTools
ex.Message, "Moving manifests")
End Try

Try
' Make sure the database temp folder exists before deleting
' the database dir.
' It might not exist if the user is running guinget offline,
' in which case the database cache will just be loaded from
' disk and won't be updated.
Await Task.Run(Sub()
If System.IO.Directory.Exists(DatabaseDir) AndAlso IO.Directory.Exists(DatabaseTempDir & "\source\Public") Then
System.IO.Directory.Delete(DatabaseDir, True)
End If
End Sub)
Try
' Make sure the database temp folder exists before deleting
' the database dir.
' It might not exist if the user is running guinget offline,
' in which case the database cache will just be loaded from
' disk and won't be updated.
Await Task.Run(Sub()
If System.IO.Directory.Exists(DatabaseDir) AndAlso IO.Directory.Exists(DatabaseTempDir & "\source\Public") Then
System.IO.Directory.Delete(DatabaseDir, True)
End If
End Sub)

' Move the database to its proper
' folder rather than copy so it's
Expand All @@ -553,17 +553,17 @@ Public Class PackageListTools
End Sub)

Catch ex As System.IO.DirectoryNotFoundException
MessageBox.Show("Couldn't find " & DatabaseTempDir & "\source\Public" & vbCrLf &
MessageBox.Show("Couldn't find " & DatabaseTempDir & "\source\Public" & vbCrLf &
"Please close any Explorer windows that may be open in this directory, and try again.",
"Moving manifests")
Catch ex As System.IO.IOException
MessageBox.Show("Please close any Explorer windows that may be open in this directory, and try again." & vbCrLf &
Catch ex As System.IO.IOException
MessageBox.Show("Please close any Explorer windows that may be open in this directory, and try again." & vbCrLf &
vbCrLf &
"Details:" & vbCrLf &
ex.Message, "Moving manifests")
End Try
End Try

Else
Else

' The calling app wants to use Robocopy.
' Partially copying code from update-manifests.bat.
Expand Down Expand Up @@ -603,12 +603,12 @@ Public Class PackageListTools
' If there's an issue deleting it here, exit the function.
Exit Function
End If
' winget-db
If Await DeleteTempDirAsync("winget-db") = False Then
' If there's an issue deleting it here, exit the function.
Exit Function
' winget-db
If Await DeleteTempDirAsync("winget-db") = False Then
' If there's an issue deleting it here, exit the function.
Exit Function
End If
End If
End If

' End checking if user clicked Cancel in the extracting phase.
End If
Expand Down Expand Up @@ -659,7 +659,6 @@ Public Class PackageListTools
' Take the Id string for each package file and append it to the
' package list array variable.
For Each PackageManifest As String In My.Computer.FileSystem.GetFiles(ManifestAppDataFolder, FileIO.SearchOption.SearchAllSubDirectories, "*.yaml")

' Append the current package manifest's path to the ManifestPath string.
Await Task.Run(Sub()
ManifestPath.Add(PackageManifest)
Expand Down Expand Up @@ -731,7 +730,7 @@ Public Class PackageListTools
#End Region

#Region "Get package details table from SQLite database"
Public Shared Function GetPackageDetailsTableFromSqliteDB() As DataTable
Public Shared Async Function GetPackageDetailsTableFromSqliteDBAsync() As Task(Of DataTable)
' Trying to load the package list as shown in this SO
' question that has the solution with it:
' https://stackoverflow.com/q/19553165
Expand Down Expand Up @@ -799,15 +798,29 @@ WHERE
' for separation.

' Column 0 is ID, 4 is Name, 2 is Version, and 6 is latest version.
packageArray.Rows.Add(SqlDataReader.GetValue(0), SqlDataReader.GetValue(4), SqlDataReader.GetValue(2), SqlDataReader.GetValue(6))
' Running this as async.
Await Task.Run(Sub()
packageArray.Rows.Add(SqlDataReader.GetValue(0), SqlDataReader.GetValue(4), SqlDataReader.GetValue(2), SqlDataReader.GetValue(6))
End Sub)
Loop
End If


'End the connection
' End the connection
' Trying to make sure the database is disconnected.
SqlDataReader.Dispose()
SqlConnection.Dispose()
SqlCommand.Dispose()
SqlDataReader.Close()
SqlConnection.Close()

' Also have it clear all pools, as it'll otherwise keep the file
' in memory in case you want to use it later. We don't need that,
' because the data is copied into memory.
' Learned about this from various answers to this SO post:
' https://stackoverflow.com/questions/8511901/system-data-sqlite-close-not-releasing-database-file
SqliteConnection.ClearAllPools()

' Return the list of packages.
Return packageArray
End Function
Expand Down
Loading

0 comments on commit b2549b4

Please sign in to comment.