Sadly it's not that easy. You can have attributes and it's not a variant. There's an option on creating attributes, that it never creates a variant. I'm now guessing that the first record that is created is the base product, so I'm searching and ordering by ID ASC and using the first one.

Yes, there's an option not to create variants, or to only create them when you place an order.

Are you saying that there are records on product.product for attributes that aren't variants? Can you give an example?

You're kinda right. Attributes that do not create variants have no records in the product_attribute_value_product_product_rel table but only the product template has records for lines. Records are in the product_template_attribute_value table. The ones that are on the line of the product template and the product is not a variant.