Replies: 2 comments 5 replies
-
|
Hi, this feature is not currently supported, we currently save the user quota, we should record all the transferred bytes instead and the deny login once the configured quota is exceeded and then reset the counter once the observation period ends. This is doable but it requires a discrete effort, a sponsorship could help. Thank you |
Beta Was this translation helpful? Give feedback.
-
|
Is this functionality possible through a plugin? I was planning on funding the development of one, which I could open-source. I would also be willing to fund you directly if this is a feature you would consider implementing. Let me know. How difficult do you think it would be, off the top of your head? Have a great day |
Beta Was this translation helpful? Give feedback.
-
I think this feature can be added directly in the SFTPGo codebase, at least at a basic level. It is not something that we can implement in one day, it requires a discrete effort and we must define the implementation details for a precise estimation. Can you please better explain your use case? Do you need these limits as bytes and number of files and for both uploads and downloads? I suppose the bytes should be incremented for both successfull and failed uploads/downloads. The number of files is more opinable, for example if a file is overwritten I suppose the file conuter should be incremented but what about if an upload or a download fails? For uploads we could increment the counter if we have a partial file on disk. For downloads, a failed download will produce a corrupted file and so we probably should not increment the counter in this case. Parallel uploads/downloads could escape the limit, as for the current quotas, but this should be an acceptable edge case. Another tricky part is when to reset the counters: if we have 100 users I want to avoid 100 goroutines with a ticker, we could simply check the elapsed time from the counters start before each download/upload and reset it if appropriate. In this case the reset time will not be exactly at the midnight (if you set a day as observation time) we store the time of the first download/upload after a reset, so it will be variable. Another approach is to simply keeps and increase the counters inside SFTPGo and reset them using the REST API with an external cron job. We could start this way but it mostly depends on your use case. For the error notifications via email etc I could write a custom hook or plugin for you: the hook/plugin already receive a specific error code if the quota is exceeded so we can implement this logic externally. Do you also need some sort of reports? For example the bytes downloaded in the last hour or day? A feature like this will increase the complexity. |
Beta Was this translation helpful? Give feedback.
-
|
My use case is that I want to use it for a beta product of a small MSP that sells storage. The customer will reserve, for example, 1TB of storage that includes 1TB of transfer per month. I need to be able to set, view, and manage that quota from the API, and in my case, I'd like to store that info in our database along with the amount of credit the customer has, etc. In my case, number of files does that matter to me though I could see how that would be useful. Again, I am only worried about download traffic, but the ability to regulate upload would also come in handy for many other use-cases (potentially mine). It would also be helpful to fight abuse to set it higher than any legitimate customer would need. As for failed uploads and downloads, does SSH produce a distinct signal on a failed transfer? If that's the case, then I would likely not count them towards the total, or provide the option to not count them, as long as the server deletes them and the user has no access to the "failed" uploads. Can you explain why parallel transfers would escape the quota? This would be a time when an in-memory database like Redis would shine. Thinking out loud here: For every open connection, you could create a line in the database and begin to track how much is transferred every ~3 seconds. Once the connection is closed, sftpgo reads the database, adds up the total transfer for the connection, writes that to the database in another place, and wipes those logs. This total would be accessible via the API and easy to integrate into an abuse/billing management system. I like your idea about a check to reset the quota when a download/upload begins. It should also be checkable and changeable via the API. That way you are not running potentially thousands of goroutines that could hit all at the same time. In the meantime, I do not mind managing the quotas entirely via the API, in fact, that would likely be preferable as I can put that load on other systems. I would require data about how an individual account has downloaded/uploaded in a day/week/month to be viewable by the customer in a dashboard I am building. If that were queriable via the API, that would be excellent. If there are alternative ways to achieve this via API that I am not seeing, I am flexible! Thank you for your time and attention. Edit: It would also be nice to be able to exclude certain IP addresses or ranges from the quota, but that is not critical. |
Beta Was this translation helpful? Give feedback.
-
|
Hi, I think I could add to the SFTPGo user profile these new fields:
The counters (used_* fields) can be modified via REST API, so you can reset them on daily, weekly and monthly basis yourself using a cronjob or such. I could also add per-IP overrides to the above limits. So you can, for example, disable the limits for a particular IP or range To allow you to populate your dashboard with data about how an individual account has downloaded/uploaded in a day/week/month we need to save something like this for each transfer
I don't want something like this in the core SFTPGo but this could be done using a custom hook/plugin. So you will have the data directly in your database and you can execute the required aggregation queries yourself. For status = 3 the plugin could send a notification email if you need this feature. I could also develop this hook/plugin for you, it depends on the sponsorship level. What database engine do you plan to use? This should cover all your requirements. If this is ok you can contact me to email address you see in each commit to reach an agreement. If you are going to use SFTPGo in your product a monthly sponsorship would be great. Thanks |
Beta Was this translation helpful? Give feedback.
-
|
Hi Drakken, I am interested in what you have proposed and will send you an email regarding moving forward. I plan on using either CochroachDB or MySQL because Redis does not appear to be supported. Can you confirm that is correct? Have a great day |
Beta Was this translation helpful? Give feedback.
-
I confirm: CochroachDB, MySQL and PostgreSQL are ok for SFTPGo core. For a custom plugin we can also use Redis if you think it will be simpler for you to query Redis to obtain your dashboard data
|
Beta Was this translation helpful? Give feedback.
-
Hello,
Thank you so much for this project! I am considering using this in a bandwidth-sensitive environment and while it is helpful to be able to limit the transfer rate of a user, it would also be awesome to be able to limit their total transfer per day/week/month/year/etc.
If I am not being clear, I am requesting the ability to specify that a user can only download 100GB per day, after that you can cut them off, send an email to the admin/user, cut their bandwidth, etc.
If this is currently possible, please let me know. I was hoping that it would be possible to develop this via the API but that does not seem to be possible because there is no option to return the user's current bandwidth.
Material reviewed: All of the docs on Github and the API documentation.
Beta Was this translation helpful? Give feedback.
All reactions