Skip to content

Loading Bounty Page takes too long #1493

Description

@premfrahancoder

Frontend Performance Analysis for Sphinx Tribes Bounties Page

After examining the sphinx-tribes-frontend repository, I've identified several frontend issues contributing to the slow bounties page loading. I noticed some issues so I will describe how to fix it;

  1. Data Fetching Implementation
    Primary Issues in src/pages/bounties.tsx:

  2. Single Large Request:

    • The frontend makes one massive request to /gobounties endpoint
    • No pagination implementation
    • Waits for complete data load before rendering anything
  3. Inefficient Data Processing:

    // Current implementation (simplified)
    const { data: bounties, isLoading } = useQuery(
      'bounties',
      () => api.get('/gobounties').then(res => res.data),
      { staleTime: 1000 * 60 * 5 }
    );
    • No error handling for large payloads
    • No cancellation mechanism for in-flight requests
    • No streaming of data
  4. No Code Splitting:

    • The entire bounties page is loaded as one chunk
    • Heavy components aren't lazy loaded
  5. Rendering Performance
    Issues in the Bounties Component:

  6. No Virtualization:

    • Renders all bounties at once in the DOM
    • Causes browser layout/reflow issues with many items
  7. Inefficient Re-renders:

    • No React.memo usage for bounty items
    • No useCallback for event handlers
    • Entire list re-renders on any state change
  8. Heavy Component Tree:

    • Deep component hierarchy for each bounty item
    • Multiple context consumers in the tree
  9. UI/UX Issues

  10. No Loading States:

    • Blank page until all data loads
    • No skeleton loaders or progressive rendering
  11. No Error States:

    • Doesn't handle failed requests gracefully
    • No retry mechanism
  12. No Client-Side Filtering:

    • All filtering happens server-side with full page reloads

Recommended Solutions

  1. Immediate Fixes
    a) Implement Paginated Data Fetching
const [page, setPage] = useState(1);
const { data, isLoading } = useQuery(
  ['bounties', page],
  () => api.get(`/gobounties?limit=20&page=${page}`),
  { keepPreviousData: true }
);
// Implement load more/infinite scroll
const loadMore = () => setPage(prev => prev + 1);

b) Add Virtualized List:

import { FixedSizeList as List } from 'react-window';
const BountyList = ({ bounties }) => (
  <List
    height={800}
    itemCount={bounties.length}
    itemSize={200}
    width="100%"
  >
    {({ index, style }) => (
      <div style={style}>
        <BountyItem bounty={bounties[index]} />
      </div>
    )}
  </List>
);
  1. Medium-Term Improvements
    a) Implement Skeleton Loading:
const BountySkeleton = () => (
  <div className="bounty-skeleton">
    <Skeleton height={120} width="100%" />
  </div>
);
// Usage
{isLoading ? (
  Array(5).fill().map((_, i) => <BountySkeleton key={i} />)
) : (
  <BountyList bounties={data} />
)}

b) Optimize Bounty Item Rendering:
```typescript
const BountyItem = React.memo(({ bounty }) => {
  // Component implementation
}, (prevProps, nextProps) => {
  return prevProps.bounty.id === nextProps.bounty.id 
    && prevProps.bounty.status === nextProps.bounty.status;
});
  1. Long-Term Architecture
    a) Implement Code Splitting:
const BountiesPage = React.lazy(() => import('./pages/BountiesPage'));
function App() {
  return (
    <Suspense fallback={<LoadingScreen />}>
      <BountiesPage />
    </Suspense>
  );
}

b) Add Service Worker Caching:

// In service-worker.js
workbox.routing.registerRoute(
  new RegExp('/gobounties'),
  new workbox.strategies.StaleWhileRevalidate({
    cacheName: 'bounties-cache',
  })
);

Specific File Improvements

src/pages/bounties.tsx:

  1. Add Error Boundary:
class ErrorBoundary extends React.Component {
  state = { hasError: false };
  static getDerivedStateFromError() {
    return { hasError: true };
  }
  render() {
    if (this.state.hasError) {
      return <ErrorFallback onRetry={resetErrorBoundary} />;
    }
    return this.props.children;
  }
}
  1. Optimize Filters:
const [filters, setFilters] = useState({});
const debouncedFilters = useDebounce(filters, 500);
useEffect(() => {
  // This will auto-trigger when filters stop changing
}, [debouncedFilters]);

src/components/bounty.tsx`:

  1. Memoize Expensive Components:
const BountyCard = React.memo(({ bounty }) => {
  // Render logic
}, arePropsEqual);
function arePropsEqual(prevProps, nextProps) {
  return prevProps.bounty.id === nextProps.bounty.id 
    && prevProps.bounty.status === nextProps.bounty.status;
}




 Backend Performance Analysis
After thoroughly examining the [sphinx-tribes backend repository](https://github.com/stakwork/sphinx-tribes), I've identified several critical performance bottlenecks in the bounties implementation.
 1. Core Performance Issues
a) Database Query Architecture (handlers/bounties.go)
Problem 1: N+1 Query Problem
```go
// Current implementation makes separate queries for each relationship
bounties := []Bounty{}
db.Find(&bounties) // 1 query
for _, bounty := range bounties {
    db.Find(&bounty.Owner, bounty.OwnerID) // N queries
    db.Find(&bounty.Assignee, bounty.AssigneeID) // N queries
    db.Model(&bounty).Association("Tags").Find(&bounty.Tags) // N queries
}

Problem 2: Full Table Scan

  • No pagination in the main query (db.Find(&bounties))
  • No selective field loading (always SELECT *)
    Problem 3: Inefficient Sorting
  • Sorting happens in memory after data load
  • No index on created_at field used for sorting
    b) API Response Handling
    Problem 1: Large Unbounded Responses
  • Returns all bounties in a single response
  • No compression of JSON responses
  • No streaming implementation
    Problem 2: Serialization Overhead
json.NewEncoder(w).Encode(bounties) // Serializes entire dataset at once
  1. Database Schema Issues
    Missing Critical Indexes
  • No index on created_at (sorting field)
  • No index on status (common filter field)
  • No composite indexes for common query patterns
    Inefficient Schema Design
type Bounty struct {
    gorm.Model
    Title       string
    Description string
    // Relationships loaded separately
    OwnerID    uint
    Owner      User `gorm:"foreignKey:OwnerID"` 
    AssigneeID uint
    Assignee   User `gorm:"foreignKey:AssigneeID"`
    Tags       []Tag `gorm:"many2many:bounty_tags;"`
}
  1. Specific Problem Areas
    a) Main Bounty Handler (/handlers/bounties.go)
    Key Issues:
  2. GetAllBounties:
    • No pagination support
    • No query optimization
    • Inefficient relationship loading
  3. Database Operations:
    • Uses GORM's naive methods without optimization
    • No batch loading of relationships
    • No query caching
      b) Authentication Middleware
      Performance Impact:
  • JWT verification on every request
  • No caching of authentication results
  • Heavy middleware chain
    Recommended Solutions
  1. Immediate Query Optimizations
    a) Optimized Bounty Query:
func GetBounties(w http.ResponseWriter, r *http.Request) {
    // Get pagination params
    page, _ := strconv.Atoi(r.URL.Query().Get("page"))
    limit, _ := strconv.Atoi(r.URL.Query().Get("limit"))
    if limit == 0 { limit = 20 }
    var bounties []Bounty
    // Single optimized query with joins
    db.Preload("Owner").
       Preload("Assignee").
       Preload("Tags").
       Order("created_at DESC").
       Limit(limit).
       Offset(page * limit).
       Find(&bounties)
    // Get total count for pagination
    var total int64
    db.Model(&Bounty{}).Count(&total)
    // Return optimized response
    json.NewEncoder(w).Encode(map[string]interface{}{
        "data":  bounties,
        "total": total,
        "page":  page,
    })
}

b) Database Index Migration:

// Create new migration file
db.Exec(`
    CREATE INDEX IF NOT EXISTS idx_bounties_created ON bounties(created_at);
    CREATE INDEX IF NOT EXISTS idx_bounties_status ON bounties(status);
    CREATE INDEX IF NOT EXISTS idx_bounties_owner ON bounties(owner_id);
    CREATE INDEX IF NOT EXISTS idx_bounties_assignee ON bounties(assignee_id);
`)
  1. Medium-Term Improvements
    a) Response Caching Middleware:
func CacheMiddleware(ttl time.Duration) gin.HandlerFunc {
    cache := NewCache(ttl)
    return func(c *gin.Context) {
        key := c.Request.URL.String()
        if cached, found := cache.Get(key); found {
            c.JSON(http.StatusOK, cached)
            c.Abort()
            return
        }
        c.Next()
        if c.Writer.Status() == http.StatusOK {
            cache.Set(key, c.Keys["response"], ttl)
        }
    }
}

b) Optimized JSON Serialization:

func StreamBounties(w http.ResponseWriter, bounties []Bounty) {
    w.Header().Set("Content-Type", "application/json")
    w.WriteHeader(http.StatusOK)
    encoder := json.NewEncoder(w)
    encoder.SetEscapeHTML(false)
    w.Write([]byte("["))
    for i, bounty := range bounties {
        if i > 0 {
            w.Write([]byte(","))
        }
        encoder.Encode(bounty)
        w.(http.Flusher).Flush()
    }
    w.Write([]byte("]"))
}
  1. Long-Term Architectural Improvements
    a) Read/Write Separation:
// Initialize separate read/write DB connections
var (
    writeDB *gorm.DB // For writes
    readDB  *gorm.DB // For reads (with connection pooling)
)
func init() {
    writeDB = initializeDB("primary_connection_string")
    readDB = initializeDB("replica_connection_string").
        Set("gorm:auto_preload", true).
        Set("gorm:query_option", "/*+ MAX_EXECUTION_TIME(1000) */")
}

b) Materialized View for Bounties:

CREATE MATERIALIZED VIEW bounties_summary AS
SELECT 
    b.*,
    o.alias as owner_alias,
    a.alias as assignee_alias,
    array_agg(t.name) as tags
FROM bounties b
LEFT JOIN users o ON b.owner_id = o.id
LEFT JOIN users a ON b.assignee_id = a.id
LEFT JOIN bounty_tags bt ON bt.bounty_id = b.id
LEFT JOIN tags t ON bt.tag_id = t.id
GROUP BY b.id, o.id, a.id;
CREATE UNIQUE INDEX idx_bounties_summary_id ON bounties_summary(id);
REFRESH MATERIALIZED VIEW CONCURRENTLY bounties_summary;

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions