Overwintering herb plants means that you bring the plants indoors so they continue growing over winter and spring. The key is to make sure the room is warm and sunny. I overwinter my herb plants on a plant stand in a window facing toward the south.
Because of limited space, I like to do this for annual, hybrids and tender perennial herb plants that I can’t grow easily from seed. For instance, I brought in my mushroom herb plant indoors because it is hard to find locally in late spring. Other good candidates for overwintering are the olive herb and African blue basil.  I actually just started overwintering some herb plants last year during Covid because it was so hard to find these herbs.