It's oak, but I didn't notice very much pepper aroma.
BTW Avery Island isn't really an island, it's a salt dome. There are beds of salt, and they're lighter than the sediments around them, so they rise as domes of salt. Oil comes up along the edges.
They keep a close eye on that place, you can't just wander around the property. Their "factory tour" is just a viewing of part of their packing line. Their real production is under wraps. They have that store, and a bit of an amusement park. It's worth visiting if you're nearby.