You can use functions like NOW() and DAYS() to test how long ago the toggle was turned on.
To do this, you need two things:
A trigger formula that will update itself whenever the toggle is turned on, just a simple formula like NOW() if $Toggle else value that is triggered on every Toggle column change.
A formula column that checks how long ago the toggle was set, with a formula like: DAYS($Last_on, NOW()) > 120
Now you will be able to filter those records based on this column. Formula columns with NOW() or TODAY() are recalculated every hour automatically, so there is no need to update records by hand.
Unfortunately, there is no way to turn off the toggle column automatically. You can use conditional formatting for some indication to guide the user that he needs to toggle this checkbox manually after the check is done.