Knowing in which page a string is found

[GiooiG]

Hi all,

as school we are given a collated/merged document related to students's reports. Each report is two or three pages long (sometimes four) and hence without automation the only way to split them correctly is to go throug hthe document and annotate the pages where each document starts. We have already extract the full text and thanks God students's names are not split on multiple lines and hence we can search for them. Each student's name is located at the beginning of the his/her won report and hence by knowing which page each name is located in we may find the series of starting page that may help us in correctly splitting the big document into single ones.

Of course we have already asked no to get such a long doc but it seems the format is not under the sender's control.

Is there a way in general to know the page where a string in a pdf doc is found? In case which functions should we use?

TIA

[JorjMcKie]

Hi,

there are several alternatives for how to do that. Let's see how we can find the best way to get your task done. Let me confirm first, that I understood your problem:

• You have a large (PDF ?) document that resulted by joining separate smaller documents, each of which has originally made a student.
• Now you want to split that large document again so you have a separate one for each student. Or at least you need to know which of the large report's pages can be tied back to the student creating them.
• • I also assume that every page in the document belongs to exactly one student: there are no pages with the upper half belonging to student n, and the lower half to student n+1, or so.

So far ok?

If that is the case, you do not need to extract text, actually.
I presume you have a list of student names to be found in the document. Then, for each name in that list, walk through the whole document and search for the name and store the encountered page number in the student name list.
When done, sort the list by ascending page numbers.
Then walk through this sorted list and for each item extract the pages starting with the item's page number and ending with the next item's page number -1 into a separate PDF.
That is actually it. Here is a snippet:

names_list = ["name1", "name2", ...]  # list of students' names
doc = fitz.open("large-report.pdf")
for i, name in enumerate(names_list):
    for page in doc:
        if page.searchFor(name) != []:  # name is on that page!
            names_list[i] = (page.number, name)  # replace entry augmented by page number
            break  # we are not interested in other pages now
names_list.sort()  # sort by page number
names_count = len(names_list)
for i in range(names_count):
    pdfout = fitz.open()
    f = names_list[i][0]  # start page number
    name = names_list[i][1]  # student name
    if i < names_count - 1:  # not the last item
        t = names_list[i+1][0]  - 1 # last page for this student
    else:
        t = names_count - 1
    pdfout.insertPDF(doc, from_page=f, to_page=t)
    pdfout.save(name + ".pdf")  # store sub doc under student's name

[GiooiG]

Hi Jori,

many thanks for your reply and Merry Christmas!
Indeed I need not to extract text but I need to know that I have found the student's name on a specific page.
As to your questions:

You have a large (PDF ?) document that resulted by joining separate smaller documents, each of which has originally made a student.

Yes! A very large PDF, 500+ pages.

Now you want to split that large document again so you have a separate one for each student. Or at least you need to know which of the large report's pages can be tied back to the student creating them.

Exactly I have to end up with the following:

1st Student's name found first on page 1, his report ends on the 2nd page
2nd Student's name found first on page 3, her report ends on the 4th page
3rd Student's name found first on page 5, her report ends on the 7th page
4th Student's name found first on page 8, his report ends on the 10th page
... (and so on)
n-th Student's name found first on page 433, her report ends on the 434th page
...
As you can see I cannot say anything upfront on which page each student's name is seen first nor the psge his/her report ends with.

I also assume that every page in the document belongs to exactly one student: there are no pages with the upper half belonging to student n, and the lower half to student n+1, or so.

Eventually I have to end up with a list or dict where I map each student's name with the page on the document his/her report starts. This way I can also say in which page the reports ends.

I had a look at your script and I think it's correct. I would only use the try/except paradigm for the if/else just to get practiced on that (I still have to understand it completely).
I would have another question if you wouldn't mind: may I get a faster execution if I did the following:

    names_list = ["name1", "name2", ...]  # list of students' names
    doc = fitz.open("large-report.pdf")
    index=0
    for page in doc:
        if page.searchFor(names_list[index]) != []:  # name is on that page!
            names_list[index] = (page.number, name)
            index+=1
    names_list.sort()  # sort by page number
    names_count = len(names_list)

?

I assume pages are structured in a way where they are checked in sequential order in the doc but I'm not sure.
Moreover, though each time I have a hit I replace an integer with a tuple I don't think the code is going to break because I move through the list in order hence there won't be scenarios where page.searchFor(names_list[index]) gets a tuple instead of a string.
Would that be a good practice?
Would you have any comment on this?

Many thanks for your thoroughful help I just needed to know the function page.searchFor but your snippet rose a question on my side and I'm happy to discuss it.

Thanks and have a nice Christmas' season,

Gio

P.S. I played a bit but I did not find a way to have my code snippet coloured as yours is, how to do that? Thanks!

[JorjMcKie]

P.S. I played a bit but I did not find a way to have my code snippet coloured as yours is, how to do that? Thanks!

Add the keyword "python" after the 3 initial ` characters. This will invoke Python syntax colouring.

[JorjMcKie]

Ok, thanks for your clarifications and assertions.
Thinking more about your case:
For performance reasons, it is probably better not to use page.searchFor(), because a lot is going on in this method under the hood. We should instead extract a page's text and search for student names in the returned string - should be much faster.
Also, once we have spotted the start page of a student, we should remove his / her entry in the names list: the list will thus become shorter every time, and it will also prevent complications, if that same student's name happens to occur again later ... one never knows.

Here is a script that works this way:

• takes a list of student names
• loops through pages of report and makes a string of page text
• checks if student name on page
• if yes, remove student from name list and store name plus page number
• at end, complete the name-page-list with end page number
• report student name, from page, to page
• report any student names not in the report

reporter.zip

[GiooiG]

Thanks Jori,

Thanks for thinking over the script, I will give a try to the new version when I am able to access the documents.
For the moment I can only thank you a lot :-)

Gio