Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding support for storing snapshots with any time resolution in CSV files and File System Repository. #268

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 11 additions & 11 deletions asset/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ The information provided on this project is strictly for informational purposes
- [func SnapshotsAsOpenings\(snapshots \<\-chan \*Snapshot\) \<\-chan float64](<#SnapshotsAsOpenings>)
- [func SnapshotsAsVolumes\(snapshots \<\-chan \*Snapshot\) \<\-chan float64](<#SnapshotsAsVolumes>)
- [type FileSystemRepository](<#FileSystemRepository>)
- [func NewFileSystemRepository\(base string\) \*FileSystemRepository](<#NewFileSystemRepository>)
- [func NewFileSystemRepository\(base string, csvOptions ...helper.CsvOption\[Snapshot\]\) \*FileSystemRepository](<#NewFileSystemRepository>)
- [func \(r \*FileSystemRepository\) Append\(name string, snapshots \<\-chan \*Snapshot\) error](<#FileSystemRepository.Append>)
- [func \(r \*FileSystemRepository\) Assets\(\) \(\[\]string, error\)](<#FileSystemRepository.Assets>)
- [func \(r \*FileSystemRepository\) Get\(name string\) \(\<\-chan \*Snapshot, error\)](<#FileSystemRepository.Get>)
Expand Down Expand Up @@ -183,7 +183,7 @@ func SnapshotsAsVolumes(snapshots <-chan *Snapshot) <-chan float64
SnapshotsAsVolumes extracts the volume field from each snapshot in the provided channel and returns a new channel containing only those volume values.The original snapshots channel can no longer be directly used afterward.

<a name="FileSystemRepository"></a>
## type [FileSystemRepository](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L20-L23>)
## type [FileSystemRepository](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L20-L26>)

FileSystemRepository stores and retrieves asset snapshots using the local file system.

Expand All @@ -194,16 +194,16 @@ type FileSystemRepository struct {
```

<a name="NewFileSystemRepository"></a>
### func [NewFileSystemRepository](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L27>)
### func [NewFileSystemRepository](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L30>)

```go
func NewFileSystemRepository(base string) *FileSystemRepository
func NewFileSystemRepository(base string, csvOptions ...helper.CsvOption[Snapshot]) *FileSystemRepository
```

NewFileSystemRepository initializes a file system repository with the given base directory.
NewFileSystemRepository initializes a file system repository with the given base directory and the CSV options.

<a name="FileSystemRepository.Append"></a>
### func \(\*FileSystemRepository\) [Append](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L92>)
### func \(\*FileSystemRepository\) [Append](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L96>)

```go
func (r *FileSystemRepository) Append(name string, snapshots <-chan *Snapshot) error
Expand All @@ -212,7 +212,7 @@ func (r *FileSystemRepository) Append(name string, snapshots <-chan *Snapshot) e
Append adds the given snapshows to the asset with the given name.

<a name="FileSystemRepository.Assets"></a>
### func \(\*FileSystemRepository\) [Assets](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L34>)
### func \(\*FileSystemRepository\) [Assets](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L38>)

```go
func (r *FileSystemRepository) Assets() ([]string, error)
Expand All @@ -221,7 +221,7 @@ func (r *FileSystemRepository) Assets() ([]string, error)
Assets returns the names of all assets in the repository.

<a name="FileSystemRepository.Get"></a>
### func \(\*FileSystemRepository\) [Get](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L56>)
### func \(\*FileSystemRepository\) [Get](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L60>)

```go
func (r *FileSystemRepository) Get(name string) (<-chan *Snapshot, error)
Expand All @@ -230,7 +230,7 @@ func (r *FileSystemRepository) Get(name string) (<-chan *Snapshot, error)
Get attempts to return a channel of snapshots for the asset with the given name.

<a name="FileSystemRepository.GetSince"></a>
### func \(\*FileSystemRepository\) [GetSince](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L61>)
### func \(\*FileSystemRepository\) [GetSince](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L65>)

```go
func (r *FileSystemRepository) GetSince(name string, date time.Time) (<-chan *Snapshot, error)
Expand All @@ -239,7 +239,7 @@ func (r *FileSystemRepository) GetSince(name string, date time.Time) (<-chan *Sn
GetSince attempts to return a channel of snapshots for the asset with the given name since the given date.

<a name="FileSystemRepository.LastDate"></a>
### func \(\*FileSystemRepository\) [LastDate](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L75>)
### func \(\*FileSystemRepository\) [LastDate](<https://github.com/cinar/indicator/blob/master/asset/file_system_repository.go#L79>)

```go
func (r *FileSystemRepository) LastDate(name string) (time.Time, error)
Expand Down Expand Up @@ -476,7 +476,7 @@ Snapshot captures a single observation of an asset's price at a specific moment.
```go
type Snapshot struct {
// Date represents the specific timestamp.
Date time.Time `format:"2006-01-02"`
Date time.Time

// Open represents the opening price for the
// snapshot period.
Expand Down
14 changes: 9 additions & 5 deletions asset/file_system_repository.go
Original file line number Diff line number Diff line change
Expand Up @@ -20,13 +20,17 @@ import (
type FileSystemRepository struct {
// base is the root directory where asset snapshots are stored.
base string

// csvOptions are the CSV options used for reading and writing snapshots.
csvOptions []helper.CsvOption[Snapshot]
}

// NewFileSystemRepository initializes a file system repository with
// the given base directory.
func NewFileSystemRepository(base string) *FileSystemRepository {
// the given base directory and the CSV options.
func NewFileSystemRepository(base string, csvOptions ...helper.CsvOption[Snapshot]) *FileSystemRepository {
return &FileSystemRepository{
base: base,
base: base,
csvOptions: csvOptions,
}
}

Expand Down Expand Up @@ -54,7 +58,7 @@ func (r *FileSystemRepository) Assets() ([]string, error) {

// Get attempts to return a channel of snapshots for the asset with the given name.
func (r *FileSystemRepository) Get(name string) (<-chan *Snapshot, error) {
return helper.ReadFromCsvFile[Snapshot](r.getCsvFileName(name), true)
return helper.ReadFromCsvFile[Snapshot](r.getCsvFileName(name), r.csvOptions...)
}

// GetSince attempts to return a channel of snapshots for the asset with the given name since the given date.
Expand Down Expand Up @@ -90,7 +94,7 @@ func (r *FileSystemRepository) LastDate(name string) (time.Time, error) {

// Append adds the given snapshows to the asset with the given name.
func (r *FileSystemRepository) Append(name string, snapshots <-chan *Snapshot) error {
return helper.AppendOrWriteToCsvFile(r.getCsvFileName(name), true, snapshots)
return helper.AppendOrWriteToCsvFile(r.getCsvFileName(name), snapshots, r.csvOptions...)
}

// getCsvFileName gets the CSV file name for the given asset name.
Expand Down
2 changes: 1 addition & 1 deletion asset/file_system_repository_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -69,7 +69,7 @@ func TestFileSystemRepositoryGetSince(t *testing.T) {
t.Fatal(err)
}

expected, err := helper.ReadFromCsvFile[asset.Snapshot]("testdata/since.csv", true)
expected, err := helper.ReadFromCsvFile[asset.Snapshot]("testdata/since.csv")
if err != nil {
t.Fatal(err)
}
Expand Down
2 changes: 1 addition & 1 deletion asset/snapshot.go
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ import (
// at a specific moment.
type Snapshot struct {
// Date represents the specific timestamp.
Date time.Time `format:"2006-01-02"`
Date time.Time

// Open represents the opening price for the
// snapshot period.
Expand Down
2 changes: 1 addition & 1 deletion asset/snapshot_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,7 @@ import (
)

func TestSnapshotsAs(t *testing.T) {
snapshots, err := helper.ReadFromCsvFile[asset.Snapshot]("testdata/repository/brk-b.csv", true)
snapshots, err := helper.ReadFromCsvFile[asset.Snapshot]("testdata/repository/brk-b.csv")
if err != nil {
t.Fatal(err)
}
Expand Down
72 changes: 56 additions & 16 deletions helper/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,7 @@ The information provided on this project is strictly for informational purposes
- [Constants](<#constants>)
- [func Abs\[T Number\]\(c \<\-chan T\) \<\-chan T](<#Abs>)
- [func Add\[T Number\]\(ac, bc \<\-chan T\) \<\-chan T](<#Add>)
- [func AppendOrWriteToCsvFile\[T any\]\(fileName string, hasHeader bool, rows \<\-chan \*T\) error](<#AppendOrWriteToCsvFile>)
- [func AppendOrWriteToCsvFile\[T any\]\(fileName string, rows \<\-chan \*T, options ...CsvOption\[T\]\) error](<#AppendOrWriteToCsvFile>)
- [func Apply\[T Number\]\(c \<\-chan T, f func\(T\) T\) \<\-chan T](<#Apply>)
- [func Buffered\[T any\]\(c \<\-chan T, size int\) \<\-chan T](<#Buffered>)
- [func ChanToJSON\[T any\]\(c \<\-chan T, w io.Writer\) error](<#ChanToJSON>)
Expand Down Expand Up @@ -69,7 +69,7 @@ The information provided on this project is strictly for informational purposes
- [func Operate3\[A any, B any, C any, R any\]\(ac \<\-chan A, bc \<\-chan B, cc \<\-chan C, o func\(A, B, C\) R\) \<\-chan R](<#Operate3>)
- [func Pipe\[T any\]\(f \<\-chan T, t chan\<\- T\)](<#Pipe>)
- [func Pow\[T Number\]\(c \<\-chan T, y T\) \<\-chan T](<#Pow>)
- [func ReadFromCsvFile\[T any\]\(fileName string, hasHeader bool\) \(\<\-chan \*T, error\)](<#ReadFromCsvFile>)
- [func ReadFromCsvFile\[T any\]\(fileName string, options ...CsvOption\[T\]\) \(\<\-chan \*T, error\)](<#ReadFromCsvFile>)
- [func Remove\(t \*testing.T, name string\)](<#Remove>)
- [func RemoveAll\(t \*testing.T, path string\)](<#RemoveAll>)
- [func RoundDigit\[T Number\]\(n T, d int\) T](<#RoundDigit>)
Expand All @@ -93,11 +93,15 @@ The information provided on this project is strictly for informational purposes
- [func \(b \*Bst\[T\]\) Remove\(value T\) bool](<#Bst[T].Remove>)
- [type BstNode](<#BstNode>)
- [type Csv](<#Csv>)
- [func NewCsv\[T any\]\(hasHeader bool\) \(\*Csv\[T\], error\)](<#NewCsv>)
- [func NewCsv\[T any\]\(options ...CsvOption\[T\]\) \(\*Csv\[T\], error\)](<#NewCsv>)
- [func \(c \*Csv\[T\]\) AppendToFile\(fileName string, rows \<\-chan \*T\) error](<#Csv[T].AppendToFile>)
- [func \(c \*Csv\[T\]\) ReadFromFile\(fileName string\) \(\<\-chan \*T, error\)](<#Csv[T].ReadFromFile>)
- [func \(c \*Csv\[T\]\) ReadFromReader\(reader io.Reader\) \<\-chan \*T](<#Csv[T].ReadFromReader>)
- [func \(c \*Csv\[T\]\) WriteToFile\(fileName string, rows \<\-chan \*T\) error](<#Csv[T].WriteToFile>)
- [type CsvOption](<#CsvOption>)
- [func WithCsvDefaultDateTimeFormat\[T any\]\(format string\) CsvOption\[T\]](<#WithCsvDefaultDateTimeFormat>)
- [func WithCsvLogger\[T any\]\(logger \*slog.Logger\) CsvOption\[T\]](<#WithCsvLogger>)
- [func WithoutCsvHeader\[T any\]\(\) CsvOption\[T\]](<#WithoutCsvHeader>)
- [type Float](<#Float>)
- [type Integer](<#Integer>)
- [type Number](<#Number>)
Expand Down Expand Up @@ -132,7 +136,7 @@ const (
CsvFormatTag = "format"

// DefaultDateTimeFormat denotes the default format of a date and time column.
DefaultDateTimeFormat = "2006-01-02 15:04:05"
DefaultDateTimeFormat = "2006-01-02"
)
```

Expand Down Expand Up @@ -182,10 +186,10 @@ fmt.Println(actual) // [2, 4, 6, 8, 10, 12, 14, 16, 18, 20]
```

<a name="AppendOrWriteToCsvFile"></a>
## func [AppendOrWriteToCsvFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L291>)
## func [AppendOrWriteToCsvFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L323>)

```go
func AppendOrWriteToCsvFile[T any](fileName string, hasHeader bool, rows <-chan *T) error
func AppendOrWriteToCsvFile[T any](fileName string, rows <-chan *T, options ...CsvOption[T]) error
```

AppendOrWriteToCsvFile writes the provided rows of data to the specified file, appending to the existing file if it exists or creating a new one if it doesn't. In append mode, the function assumes that the existing file's column order matches the field order of the given row struct to ensure consistent data structure.
Expand Down Expand Up @@ -805,10 +809,10 @@ fmt.Println(helper.ChanToSlice(squared)) // [4, 9, 25, 100]
```

<a name="ReadFromCsvFile"></a>
## func [ReadFromCsvFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L278>)
## func [ReadFromCsvFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L310>)

```go
func ReadFromCsvFile[T any](fileName string, hasHeader bool) (<-chan *T, error)
func ReadFromCsvFile[T any](fileName string, options ...CsvOption[T]) (<-chan *T, error)
```

ReadFromCsvFile creates a CSV instance, parses CSV data from the provided filename, maps the data to corresponding struct fields, and delivers it through the channel.
Expand Down Expand Up @@ -1095,7 +1099,7 @@ type BstNode[T Number] struct {
```

<a name="Csv"></a>
## type [Csv](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L40-L50>)
## type [Csv](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L40-L53>)

Csv represents the configuration for CSV reader and writer.

Expand All @@ -1109,16 +1113,16 @@ type Csv[T any] struct {
```

<a name="NewCsv"></a>
### func [NewCsv](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L54>)
### func [NewCsv](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L80>)

```go
func NewCsv[T any](hasHeader bool) (*Csv[T], error)
func NewCsv[T any](options ...CsvOption[T]) (*Csv[T], error)
```

NewCsv function initializes a new CSV instance. The parameter hasHeader indicates whether the CSV contains a header row.
NewCsv creates a new CSV instance with the provided options.

<a name="Csv[T].AppendToFile"></a>
### func \(\*Csv\[T\]\) [AppendToFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L174>)
### func \(\*Csv\[T\]\) [AppendToFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L206>)

```go
func (c *Csv[T]) AppendToFile(fileName string, rows <-chan *T) error
Expand All @@ -1127,7 +1131,7 @@ func (c *Csv[T]) AppendToFile(fileName string, rows <-chan *T) error
AppendToFile appends the provided rows of data to the end of the specified file, creating the file if it doesn't exist. In append mode, the function assumes that the existing file's column order matches the field order of the given row struct to ensure consistent data structure.

<a name="Csv[T].ReadFromFile"></a>
### func \(\*Csv\[T\]\) [ReadFromFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L150>)
### func \(\*Csv\[T\]\) [ReadFromFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L182>)

```go
func (c *Csv[T]) ReadFromFile(fileName string) (<-chan *T, error)
Expand All @@ -1136,7 +1140,7 @@ func (c *Csv[T]) ReadFromFile(fileName string) (<-chan *T, error)
ReadFromFile parses the CSV data from the provided file name, maps the data to corresponding struct fields, and delivers the resulting rows through the channel.

<a name="Csv[T].ReadFromReader"></a>
### func \(\*Csv\[T\]\) [ReadFromReader](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L95>)
### func \(\*Csv\[T\]\) [ReadFromReader](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L127>)

```go
func (c *Csv[T]) ReadFromReader(reader io.Reader) <-chan *T
Expand All @@ -1145,14 +1149,50 @@ func (c *Csv[T]) ReadFromReader(reader io.Reader) <-chan *T
ReadFromReader parses the CSV data from the provided reader, maps the data to corresponding struct fields, and delivers the resulting it through the channel.

<a name="Csv[T].WriteToFile"></a>
### func \(\*Csv\[T\]\) [WriteToFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L190>)
### func \(\*Csv\[T\]\) [WriteToFile](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L222>)

```go
func (c *Csv[T]) WriteToFile(fileName string, rows <-chan *T) error
```

WriteToFile creates a new file with the given name and writes the provided rows of data to it, overwriting any existing content.

<a name="CsvOption"></a>
## type [CsvOption](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L56>)

CsvOption represents a functional option for configuring the CSV instance.

```go
type CsvOption[T any] func(*Csv[T])
```

<a name="WithCsvDefaultDateTimeFormat"></a>
### func [WithCsvDefaultDateTimeFormat](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L73>)

```go
func WithCsvDefaultDateTimeFormat[T any](format string) CsvOption[T]
```

WithCsvDefaultDateTimeFormat sets the default date and time format for the CSV instance.

<a name="WithCsvLogger"></a>
### func [WithCsvLogger](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L66>)

```go
func WithCsvLogger[T any](logger *slog.Logger) CsvOption[T]
```

WithCsvLogger sets the logger for the CSV instance.

<a name="WithoutCsvHeader"></a>
### func [WithoutCsvHeader](<https://github.com/cinar/indicator/blob/master/helper/csv.go#L59>)

```go
func WithoutCsvHeader[T any]() CsvOption[T]
```

WithoutCsvHeader disables the header row in the CSV.

<a name="Float"></a>
## type [Float](<https://github.com/cinar/indicator/blob/master/helper/helper.go#L27-L29>)

Expand Down
Loading
Loading