You can use set_row() function for that.

Help on method set_row in module xlsxwriter.worksheet:

set_row(row, height=None, cell_format=None, options=None)
    Set the width, and other properties of a row.
    
    Args:
        row:         Row number (zero-indexed).
        height:      Row width. (optional).
        cell_format: Row cell_format. (optional).
        options:     Dict of options such as hidden, level and collapsed.
    
    Returns:
        0:  Success.
        -1: Row number is out of worksheet bounds.

Ref: https://stackoverflow.com/questions/36100593/python-xlsxwriter-change-row-height-for-all-rows-in-the-sheet