Skip to content

Commit

Permalink
getSetProps working - tests added
Browse files Browse the repository at this point in the history
  • Loading branch information
@mohammedzakikochargi
mohammedzakikochargi committed Jul 17, 2023
1 parent 5f93675 commit e3088f2
Show file tree
Hide file tree
Showing 6 changed files with 139 additions and 25 deletions.
7 changes: 6 additions & 1 deletion .vscode/settings.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -72,6 +72,11 @@
"xstring": "cpp",
"xtr1common": "cpp",
"xtree": "cpp",
"xutility": "cpp"
"xutility": "cpp",

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 17, 2023

Collaborator

why is this file committed at all ?

This comment has been minimized.

Sign in to view

Copy link
@mohammedzakikochargi

mohammedzakikochargi Jul 17, 2023

Author Collaborator

why is this file committed at all ?

By mistake, will remove it.
"*.ipp": "cpp",
"any": "cpp",
"filesystem": "cpp",
"array": "cpp",
"variant": "cpp"
}
}
1 change: 1 addition & 0 deletions base/include/Module.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -177,6 +177,7 @@ class Module {
boost::shared_ptr<PaceMaker> getPacer() { return pacer; }
static frame_sp getFrameByType(frame_container& frames, int frameType);
virtual void flushQue();
bool getPlayDirection() { return mDirection; }
virtual void flushQueRecursive();
protected:
virtual boost_deque<frame_sp> getFrames(frame_container& frames);
Expand Down
1 change: 1 addition & 0 deletions base/include/OrderedCacheOfFiles.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -48,6 +48,7 @@ class OrderedCacheOfFiles
bool fetchAndUpdateFromDisk(std::string videoFile, uint64_t& start_ts, uint64_t& end_ts);
bool fetchFromCache(std::string& videoFile, uint64_t& start_ts, uint64_t& end_ts);
void readVideoStartEnd(std::string& filePath, uint64_t& start_ts, uint64_t& end_ts);
void clearCache();
bool refreshCache();
std::string getLastVideoInCache() { return videoCache.rbegin()->path; }
void updateCache(std::string& filePath, uint64_t& start_ts, uint64_t& end_ts); // allow updates from playback
Expand Down
40 changes: 38 additions & 2 deletions base/src/Mp4ReaderSource.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -74,14 +74,49 @@ class Mp4ReaderDetailAbs
void setProps(Mp4ReaderSourceProps& props)
{

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

mProps should not be assigned before. Only when everything else is successfull
mProps = props;
mState.mVideoPath = mProps.videoPath;

mState.direction = mProps.direction;

std::chrono::time_point<std::chrono::system_clock> t = std::chrono::system_clock::now();
auto dur = std::chrono::duration_cast<std::chrono::milliseconds>(t.time_since_epoch());
uint64_t nowTS = dur.count();
reloadFileAfter = calcReloadFileAfter();
auto canonicalVideoPath = boost::filesystem::canonical(mProps.videoPath);

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

have a try catch for canonical - test with a wrong path (does not exist on the drive) - catch it and move on. Props will remain unchanged.
if (mState.mVideoPath != mProps.videoPath && mState.mVideoPath != "")
{
mState.mVideoPath = canonicalVideoPath.string();

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

For this module, remember: dont alter mState till everything else is successful
sentEOSSignal = false;

if (mProps.parseFS)

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

dont use mProps - use local props arg
{
auto boostVideoTS = boost::filesystem::path(mState.mVideoPath).stem().string();
uint64_t start_parsing_ts = 0;
try
{
start_parsing_ts = std::stoull(boostVideoTS);
}
catch (std::invalid_argument)
{
auto msg = "Video File name not in proper format.Check the filename sent as props. \
If you want to read a file with custom name instead, please disable parseFS flag.";
LOG_ERROR << msg;
throw AIPException(AIP_FATAL, msg);
}
auto skipDir = boost::filesystem::path(mState.mVideoPath).parent_path().parent_path().parent_path().string();
if (mProps.skipDir != skipDir)

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

add a comment: //if the root folder is different/changed
{

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023 •
edited
Loading

Collaborator

change this code - this is giving wrong impression to the reader
mProps.skipDir = skipDir;
cof = boost::shared_ptr<OrderedCacheOfFiles>(new OrderedCacheOfFiles(mProps.skipDir));
cof->clearCache();
cof->parseFiles(start_parsing_ts, mState.direction, true, false);
initNewVideo(true);

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

set mProps and mState.skipDir just before initNewVideo call
return;
}
cof->parseFiles(start_parsing_ts, mState.direction, true, false);
}
initNewVideo();

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

set mProps in this case & add a comment for which case is this
}
mState.mVideoPath = canonicalVideoPath.string();

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

why ?

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

you will run into problem if setProps fails
}

std::string getOpenVideoPath()
Expand Down Expand Up @@ -1417,7 +1452,8 @@ Mp4ReaderSourceProps Mp4ReaderSource::getProps()

bool Mp4ReaderSource::handlePropsChange(frame_sp& frame)
{
Mp4ReaderSourceProps props(mDetail->mProps.videoPath, mDetail->mProps.reInitInterval, mDetail->mProps.direction, mDetail->mProps.readLoop, mDetail->mProps.giveLiveTS, mDetail->mProps.parseFSTimeoutDuration, mDetail->mProps.bFramesEnabled);
bool direction = getPlayDirection();
Mp4ReaderSourceProps props(mDetail->mProps.videoPath, mDetail->mProps.reInitInterval, direction, mDetail->mProps.readLoop, mDetail->mProps.giveLiveTS, mDetail->mProps.parseFSTimeoutDuration, mDetail->mProps.bFramesEnabled);
bool ret = Module::handlePropsChange(frame, props);
mDetail->setProps(props);
//mDetail->Init();
Expand Down
9 changes: 9 additions & 0 deletions base/src/OrderedCacheOfFiles.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -803,6 +803,15 @@ void OrderedCacheOfFiles::deleteLostEntry(std::string& filePath)
return;
}

void OrderedCacheOfFiles::clearCache()
{
if (videoCache.size())
{

This comment has been minimized.

Sign in to view

Copy link
@mraduldubey

mraduldubey Jul 18, 2023

Collaborator

format properly
boost::mutex::scoped_lock(m_mutex);
videoCache.clear();
}
}

bool OrderedCacheOfFiles::refreshCache()
{
auto direction = lastKnownPlaybackDir;
Expand Down
106 changes: 84 additions & 22 deletions base/test/mp4readersource_tests.cpp
View file
Original file line number Diff line number Diff line change
Expand Up @@ -72,10 +72,10 @@ class MetadataSink : public Module
MetadataSinkProps mProps;
};

struct setupMp4ReaderTest
struct SetupMp4ReaderTest
{

setupMp4ReaderTest(std::string videoPath, framemetadata_sp inputMetadata, FrameMetadata::FrameType frameType, bool parseFS, int uniqMetadata = 0)
SetupMp4ReaderTest(std::string videoPath, framemetadata_sp inputMetadata, FrameMetadata::FrameType frameType, bool parseFS, int uniqMetadata = 0)
{
LoggerProps loggerProps;
loggerProps.logLevel = boost::log::trivial::severity_level::info;
Expand All @@ -102,12 +102,12 @@ struct setupMp4ReaderTest
metaSink = boost::shared_ptr<MetadataSink>(new MetadataSink(metaSinkProps));
mp4Reader->setNext(metaSink);

mp4Reader->init();
sink->init();
BOOST_TEST(mp4Reader->init());
BOOST_TEST(sink->init());

}

~setupMp4ReaderTest()
~SetupMp4ReaderTest()
{
mp4Reader->term();
metaSink->term();
Expand All @@ -120,16 +120,14 @@ struct setupMp4ReaderTest
boost::shared_ptr<MetadataSink> metaSink;
};

// todo - basic read test with 4 saveOrCompare for jpeg and h264 (done)
// - read metadata as well.
BOOST_AUTO_TEST_CASE(mp4v_to_jpg_frames_metadata)
{
std::string videoPath = "./data/Mp4_videos/jpg_video_metada/20230513/0019/1686666193885.mp4";
std::string videoPath = "./data/Mp4_videos/jpg_video_metadata/20230513/0019/1686666193885.mp4";
std::string outPath = "data/mp4Reader_saveOrCompare/jpeg/frame_000";
auto frameType = FrameMetadata::FrameType::ENCODED_IMAGE;
auto encodedImageMetadata = framemetadata_sp(new EncodedImageMetadata(0, 0));
bool parseFS = false;
setupMp4ReaderTest s(videoPath, encodedImageMetadata, frameType, parseFS);
SetupMp4ReaderTest s(videoPath, encodedImageMetadata, frameType, parseFS);

for (int i = 0; i < 180; i++)
{
Expand Down Expand Up @@ -165,7 +163,7 @@ BOOST_AUTO_TEST_CASE(mp4v_to_h264_frames_metadata)
bool parseFS = false;
auto h264ImageMetadata = framemetadata_sp(new H264Metadata(0, 0));
auto frameType = FrameMetadata::FrameType::H264_DATA;
setupMp4ReaderTest s(videoPath, h264ImageMetadata, frameType, parseFS);
SetupMp4ReaderTest s(videoPath, h264ImageMetadata, frameType, parseFS);

for (int i = 0; i < 180; i++)
{
Expand Down Expand Up @@ -193,7 +191,6 @@ BOOST_AUTO_TEST_CASE(mp4v_to_h264_frames_metadata)
}
}

// todo - read timestamp from file
BOOST_AUTO_TEST_CASE(read_timeStamp_from_custom_fileName)
{
/* file structure parsing test */
Expand All @@ -203,18 +200,17 @@ BOOST_AUTO_TEST_CASE(read_timeStamp_from_custom_fileName)
auto frameType = FrameMetadata::FrameType::H264_DATA;
auto h264ImageMetadata = framemetadata_sp(new H264Metadata(0, 0));
bool parseFS = false;
setupMp4ReaderTest s(videoPath, h264ImageMetadata, frameType, parseFS);
SetupMp4ReaderTest s(videoPath, h264ImageMetadata, frameType, parseFS);

s.mp4Reader->step();
auto frames = s.sink->pop();
auto frame = frames.begin()->second;
BOOST_TEST(frame->timestamp == 1673420640350);
}

// todo - one get set prop - fix the flow (done)
BOOST_AUTO_TEST_CASE(getSetProps)
BOOST_AUTO_TEST_CASE(getSetProps_change_video_path)
{
std::string videoPath = "./data/Mp4_videos/jpg_video/20220928/0013/1666943213667.mp4";
std::string videoPath = "./data/Mp4_videos/mp4_seek_tests/20220522/0023/1655919060000.mp4";
std::string outPath = "./data/testOutput/outFrames/";
bool parseFS = true;

Expand All @@ -241,7 +237,8 @@ BOOST_AUTO_TEST_CASE(getSetProps)
mp4Reader->init();
sink->init();
frame_container frames;
for (int i = 0; i < 182; i++)
// go till the second last frame
for (int i = 0; i < 1268; i++)
{
mp4Reader->step();
frames = sink->pop();
Expand All @@ -251,22 +248,87 @@ BOOST_AUTO_TEST_CASE(getSetProps)
propsChange.readLoop = true;
mp4Reader->setProps(propsChange);

//last frame of the open video
// read the second last frame of the open video
mp4Reader->step();
frames = sink->pop();
auto secondLastFrame = frames.begin()->second;
BOOST_TEST(secondLastFrame->size() == 289);
BOOST_TEST(secondLastFrame->timestamp == 1655919081120);

//change the video file path , Now read first frame new video instead of last frame of open video
propsChange = mp4Reader->getProps();
propsChange.videoPath = "./data/Mp4_videos/mp4_seek_tests/20220523/0001/1655926320000.mp4";
mp4Reader->setProps(propsChange);
mp4Reader->step();
frames = sink->pop();
auto frame = frames.begin()->second;
BOOST_TEST(frame->timestamp == 1655926320000);
}

BOOST_AUTO_TEST_CASE(getSetProps_change_root_folder)
{
std::string videoPath = "./data/Mp4_videos/h264_video_metadata/20230514/0011/1686723796848.mp4";
std::string outPath = "./data/testOutput/outFrames/";
bool parseFS = true;

LoggerProps loggerProps;
loggerProps.logLevel = boost::log::trivial::severity_level::info;
Logger::setLogLevel(boost::log::trivial::severity_level::info);
Logger::initLogger(loggerProps);

boost::filesystem::path dir(outPath);

auto mp4ReaderProps = Mp4ReaderSourceProps(videoPath, parseFS, 0, true, false, false);
auto mp4Reader = boost::shared_ptr<Mp4ReaderSource>(new Mp4ReaderSource(mp4ReaderProps));
auto h264VideoMetadata = framemetadata_sp(new H264Metadata(0, 0));
mp4Reader->addOutPutPin(h264VideoMetadata);
auto mp4Metadata = framemetadata_sp(new Mp4VideoMetadata("v_1"));
mp4Reader->addOutPutPin(mp4Metadata);

auto sink = boost::shared_ptr<ExternalSinkModule>(new ExternalSinkModule());

std::vector<std::string> encodedImagePin;
encodedImagePin = mp4Reader->getAllOutputPinsByType(FrameMetadata::H264_DATA);
mp4Reader->setNext(sink, encodedImagePin);

mp4Reader->init();
sink->init();
frame_container frames;
// go till the second last frame
for (int i = 0; i < 230; i++)
{
mp4Reader->step();
frames = sink->pop();
}

auto propsChange = mp4Reader->getProps();
propsChange.readLoop = true;
mp4Reader->setProps(propsChange);

// read the last frame of the open video
mp4Reader->step();
frames = sink->pop();
auto lastFrame = frames.begin()->second;
BOOST_TEST(lastFrame->timestamp == 1686723806278);

//change the video file path , Now read first frame new video of changed root dir instead of last frame of open video
propsChange = mp4Reader->getProps();
propsChange.videoPath = "./data/Mp4_videos/mp4_seeks_tests_h264/20230111/0012/1673420640350.mp4";
mp4Reader->setProps(propsChange);
mp4Reader->step();
frames = sink->pop();
auto frame = frames.begin()->second;
BOOST_TEST(frame->size() == 4345);
BOOST_TEST(frame->timestamp == 1673420640350);
}

BOOST_AUTO_TEST_CASE(parse_root_dir_and_find_the_video)
{
std::string videoPath = "./data/Mp4_videos/jpg_video";
std::string outPath = "data/testOutput/outFrames";
boost::filesystem::path file("frame_??????.jpg");
auto frameType = FrameMetadata::FrameType::ENCODED_IMAGE;
auto encodedImageMetadata = framemetadata_sp(new EncodedImageMetadata(0, 0));
bool parseFS = false;
setupMp4ReaderTest s(videoPath, encodedImageMetadata, frameType, parseFS);
SetupMp4ReaderTest s(videoPath, encodedImageMetadata, frameType, parseFS);

BOOST_TEST(s.mp4Reader->step());
auto frames = s.sink->pop();
Expand Down Expand Up @@ -341,7 +403,7 @@ BOOST_AUTO_TEST_CASE(mp4reader_waits_when_no_video_and_reads_whenever_video_is_w

}

// todo (done)

BOOST_AUTO_TEST_CASE(check_exposed_params)
{
std::string startingVideoPath = "data/mp4_video/mp4_seek_tests/20220522/0016/1655895162221.mp4";
Expand Down Expand Up @@ -376,7 +438,7 @@ BOOST_AUTO_TEST_CASE(check_exposed_params)
BOOST_TEST(mp4Reader->getOpenVideoFrameCount() == 181);
}

// todo (done)

BOOST_AUTO_TEST_CASE(max_buffer_size_change_props)
{
std::string startingVideoPath = "data/mp4_video/mp4_seek_tests/20220522/0016/1655895162221.mp4";
Expand Down

2 comments on commit e3088f2

@mraduldubey
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add one more test - where getSetProps fails (new root dir does not exist) & the reader should continue from the correct older state (before changed prop)

@mraduldubey
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two cases - parseFS on and parseFS off (old state)

Please sign in to comment.