Basically there are 4 steps:
Rename old table
Create new table
with partitioning
Add constraint on old table for it's proposed partition ranges
Attach old table as a partition to new partition table
Sounds easy right?
What about indexes? What about Triggers?
And guess what, there are other traps to watch out for!
-- Add the check constraint
alter table public.test_partition_10_25 add constraint test_partition_10_25_constraint CHECK (id >= 10 AND id <25) NOT VALID;
update pg_constraint pgc
SET convalidated = true
FROM pg_class c
WHERE
c.oid = pgc.conrelid
AND connamespace = 'public'::regnamespace::oid
AND c.relname = 'test_partition_10_25'
AND conname = 'test_partition_10_25_constraint';
pgloader will keep a separate file of rejected data, but continue trying to copy good data in your database.
pgloader also implements data reformatting, a typical example of that being the transformation of MySQL datestamps 0000-00-00 and 0000-00-00 00:00:00 to PostgreSQL NULL value
@Repository
public interface PostRepository extends BaseJpaRepository<Post, Long> {
@Query("""
select p
from Post p
where date(p.createdOn) >= :sinceDate
"""
)
@QueryHints(
@QueryHint(name = AvailableHints.HINT_FETCH_SIZE, value = "25")
)
Stream<Post> streamByCreatedOnSince(@Param("sinceDate") LocalDate sinceDate);
}
The FETCH_SIZE JPA query hint is necessary for PostgreSQL and MySQL to instruct the JDBC Driver to prefetch at most 25 records. Otherwise, the PostgreSQL and MySQL JDBC Drivers would prefetch all the query results prior to traversing the underlying ResultSet.