fileutil: add configurable options for line reading (trim, skip empty, comment filter, buffer size)

[nandaanomi]

Description

Add configurable options for line reading in the fileutil package.

Problem

Current ReadFile() and related functions have several limitations:

1. No trimming support - cannot automatically trim whitespace from lines
2. No skip empty lines - cannot filter out empty lines
3. No comment filtering - cannot skip lines starting with # (common in config files)
4. Poor error handling - scanner errors are silently dropped when using channel-based functions
5. No buffer size control - cannot customize scanner buffer for large lines

Proposed Solution

Add new functions and types with options pattern:

• LineOption type for configuration
• WithTrimSpace() — trim leading/trailing whitespace
• WithSkipEmpty() — skip empty lines
• WithComment(prefix string) — skip lines with a comment prefix (e.g., #)
• WithBufferSize(size int) — custom scanner buffer size

Two new functions:

1. ReadFileWithError() — reads a file with proper error propagation via two channels (lines + errors)
2. ReadLinesStream() — reads a file with all configurable options

Existing functions will remain unchanged for backward compatibility.

Benefits

• Better error handling (no more silent failures)
• More flexible file processing
• Backward compatible with existing code
• Useful for all ProjectDiscovery tools (Subfinder, HTTPX, Nuclei, Katana, etc.)
• Follows Go best practices with options pattern

Example Usage

linesCh, errCh := fileutil.ReadLinesStream("file.txt",
    fileutil.WithTrimSpace(),
    fileutil.WithSkipEmpty(),
    fileutil.WithComment("#"),
    fileutil.WithBufferSize(1024*1024),
)

for {
    select {
    case line, ok := <-linesCh:
        if !ok {
            select {
            case err, ok := <-errCh:
                if ok && err != nil {
                    return err
                }
            default:
            }
            return nil
        }
        // process line
    case err, ok := <-errCh:
        if !ok {
            return nil
        }
        return err
    }
}